I've added a lot of checks into code to prevent them looking at the wall, but it seems it's still insufficient...
They can hear the enemy throw the wall - that's the main reason they look there. Sometimes they also try to shoot throw the walls - if the wall lets so.