Hearing sounds

[stefanhendriks]

I am quite curious how you guys code your hearing of a bot. I know several methods, but i am not sure which one is the best. Like for players you can scan them, check if they are in distance and try to see if that person runs (and thus can be heared). However, i still am not quite satisfied with that, there is also another way of intercepting sounds, like firing weapons and such right?

[Rick]

hmm I do it like Count Floyd did it... I just store the sound in a array for clients when pfnEmitSound gets called. Elsewhere I check if this sound is heareble by looking how far it is and if the player who made the sound is visible

[Pierre-Marie Baty]

I think I can boast to have the most complicated sound code in my bot - ever

In fact, sounds are of 2 types:
- "important" sounds are played by the server, which orders then some or all of its client to play it at the same time, through a network message.
- most of the other sounds are emulated by the client DLL, which interprets the game and decides if a sound must be played or not. This is done to save a whole lot of network bandwidth.

Only a few sounds are hookable through server-side functions, actually. All the rest is client-side stuff only.

Since bots have no client DLL, they need to emulate all the other sounds themselves. Movement sounds AND weapon sounds. Movement sounds can be evaluated each frame, while weapon sounds can be hooked by catching "ammo decrease" network messages.

Here's how I do it (here for emulating a Counter-Strike client DLL).

void PlayClientSoundsForBots (edict_t *pPlayer)
{
   // this function determines if the player pPlayer is walking or running, or climbing a ladder,
   // or landing on the ground, and so if he's likely to emit some client sound or not. Since
   // these types of sounds are predicted on the client side only, and bots have no client DLL,
   // we have to simulate their emitting in order for the bots to hear them. So in case a player
   // is moving, we bring his footstep sounds to the ears of the bots around. This sound is based
   // on the texture the player is walking on. Using TraceTexture(), we ask the engine for that
   // texture, then look up in the step sounds database in order to determine which footstep
   // sound is related to that texture. The ladder check then assumes that a player moving
   // vertically, not on the ground, having a ladder in his immediate surroundings is climbing
   // it, and the ladder sound is emitted periodically the same way footstep sounds are emitted.
   // Then, the landing check looks for non-null value of the player's punch angles (screen
   // tilting) while this player's damage inflictor be either null, or the world. If the test
   // success, a landing sound is emitted as well.
   // thanks to Tom Simpson from FoxBot for the water sounds handling
   edict_t *pGroundEntity = NULL;
   const char *texture_name, *player_weapon;
   char texture_type;
   char sound_path[256];
   int player_index;
   float player_velocity, volume;
   player_index = ENTINDEX (pPlayer) - 1; // get the player index
   if (DebugLevel.is_observer && !players[player_index].is_racc_bot)
	  return; // skip real players if in observer mode
   player_velocity = pPlayer->v.velocity.Length (); // get the player velocity
   player_weapon = STRING (pPlayer->v.weaponmodel) + 9; // get player's weapon, skip 'models/p_'
   // does the server allow footstep sounds AND this player is actually moving
   // AND is player on the ground AND is it time for him to make a footstep sound
   // OR has that player just landed on the ground after a jump ?
   if ((server.does_footsteps && IsOnFloor (pPlayer) && (player_velocity > 0)
		&& (players[player_index].step_sound_time < *server.time))
	   || ((FNullEnt (pPlayer->v.dmg_inflictor) || (pPlayer->v.dmg_inflictor == pWorldEntity))
		   && (pPlayer->v.punchangle != g_vecZero)
		   && !(players[player_index].prev_v.flags & (FL_ONGROUND | FL_PARTIALGROUND))))
   {
	  // is this player sloshing in water ?
	  if (pPlayer->v.waterlevel > 0)
	  {
		 sprintf (sound_path, "player/pl_slosh%d.wav", RANDOM_LONG (1, 4)); // build a slosh sound path
		 // bring slosh sound from this player to the bots' ears
		 DispatchSound (sound_path, pPlayer->v.origin + Vector (0, 0, -18), 0.9, ATTN_NORM);
		 players[player_index].step_sound_time = *server.time + 0.300; // next slosh in 300 milliseconds
	  }
	  // else this player is definitely not in water, does he move fast enough to make sounds ?
	  else if (player_velocity > MAX_WALK_SPEED)
	  {
		 // get the entity under the player's feet
		 if (!FNullEnt (pPlayer->v.groundentity))
			pGroundEntity = pPlayer->v.groundentity; // this player is standing over something
		 else
			pGroundEntity = pWorldEntity; // this player is standing over the world itself
		 // ask the engine for the texture name on pGroundEntity under the player's feet
		 texture_name = TRACE_TEXTURE (pGroundEntity, pPlayer->v.origin, Vector (0, 0, -9999));
		 // if the engine found the texture, ask the game DLL for the texture type
		 if (texture_name != NULL)
			texture_type = PM_FindTextureType ((char *) texture_name); // ask for texture type
		 // given the type of texture under player's feet, prepare a sound file for being played
		 switch (texture_type)
		 {
			default:
			case CHAR_TEX_CONCRETE:
			   sprintf (sound_path, "player/pl_step%d.wav", RANDOM_LONG (1, 4)); // 4 step sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_METAL:
			   sprintf (sound_path, "player/pl_metal%d.wav", RANDOM_LONG (1, 4)); // 4 metal sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_DIRT:
			   sprintf (sound_path, "player/pl_dirt%d.wav", RANDOM_LONG (1, 4)); // 4 dirt sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_VENT:
			   sprintf (sound_path, "player/pl_duct%d.wav", RANDOM_LONG (1, 4)); // 4 duct sounds
			   volume = 0.5;
			   break;
			case CHAR_TEX_GRATE:
			   sprintf (sound_path, "player/pl_grate%d.wav", RANDOM_LONG (1, 4)); // 4 grate sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_TILE:
			   sprintf (sound_path, "player/pl_tile%d.wav", RANDOM_LONG (1, 5)); // 5 tile sounds
			   volume = 0.8;
			   break;
			case CHAR_TEX_SLOSH:
			   sprintf (sound_path, "player/pl_slosh%d.wav", RANDOM_LONG (1, 4)); // 4 slosh sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_WOOD:
			   sprintf (sound_path, "debris/wood%d.wav", RANDOM_LONG (1, 3)); // 3 wood sounds
			   volume = 0.9;
			   break;
			case CHAR_TEX_GLASS:
			case CHAR_TEX_COMPUTER:
			   sprintf (sound_path, "debris/glass%d.wav", RANDOM_LONG (1, 4)); // 4 glass sounds
			   volume = 0.8;
			   break;
			case 'N':
			   sprintf (sound_path, "player/pl_snow%d.wav", RANDOM_LONG (1, 6)); // 6 snow sounds
			   volume = 0.8;
			   break;
		 }
		 // did we hit a breakable ?
		 if (!FNullEnt (pPlayer->v.groundentity)
			 && (strcmp ("func_breakable", STRING (pPlayer->v.groundentity->v.classname)) == 0))
			volume /= 1.5; // drop volume, the object will already play a damaged sound
		 // bring footstep sound from this player's feet to the bots' ears
		 DispatchSound (sound_path, pPlayer->v.origin + Vector (0, 0, -18), volume, ATTN_NORM);
		 players[player_index].step_sound_time = *server.time + 0.3; // next step in 300 milliseconds
	  }
   }
   // is this player completely in water AND it's time to play a wade sound
   // AND this player is pressing the jump key for swimming up ?
   if ((players[player_index].step_sound_time < *server.time)
	   && (pPlayer->v.waterlevel == 2) && (pPlayer->v.button & IN_JUMP))
   {
	  sprintf (sound_path, "player/pl_wade%d.wav", RANDOM_LONG (1, 4)); // build a wade sound path
	  // bring wade sound from this player to the bots' ears
	  DispatchSound (sound_path, pPlayer->v.origin + Vector (0, 0, -18), 0.9, ATTN_NORM);
	  players[player_index].step_sound_time = *server.time + 0.5; // next wade in 500 milliseconds
   }
   // now let's see if this player is on a ladder, for that we consider that he's not on the
   // ground, he's actually got a velocity (especially vertical), and that he's got a
   // func_ladder entity right in front of him. Is that player moving anormally NOT on ground ?
   if ((pPlayer->v.velocity.z != 0) && IsFlying (pPlayer) && !IsOnFloor (pPlayer))
   {
	  pGroundEntity = NULL; // first ensure the pointer at which to start the search is NULL
	  // cycle through all ladders...
	  while ((pGroundEntity = UTIL_FindEntityByString (pGroundEntity, "classname", "func_ladder")) != NULL)
	  {
		 // is this ladder at the same height as the player AND the player is next to it (in
		 // which case, assume he's climbing it), AND it's time for him to emit ladder sound ?
		 if ((pGroundEntity->v.absmin.z < pPlayer->v.origin.z) && (pGroundEntity->v.absmax.z > pPlayer->v.origin.z)
			 && (((pGroundEntity->v.absmin + pGroundEntity->v.absmax) / 2 - pPlayer->v.origin).Length2D () < 40)
			 && (players[player_index].step_sound_time < *server.time))
		 {
			volume = 0.8; // default volume for ladder sounds (empirical)
			// now build a random sound path amongst the 4 different ladder sounds
			sprintf (sound_path, "player/pl_ladder%d.wav", RANDOM_LONG (1, 4));
			// is the player ducking ?
			if (pPlayer->v.button & IN_DUCK)
			   volume /= 1.5; // drop volume, the player is trying to climb silently
			// bring ladder sound from this player's feet to the bots' ears
			DispatchSound (sound_path, pPlayer->v.origin + Vector (0, 0, -18), volume, ATTN_NORM);
			players[player_index].step_sound_time = *server.time + 0.500; // next in 500 milliseconds
		 }
	  }
   }
   // and now let's see if this player is pulling the pin of a grenade...
   if ((pPlayer->v.button & IN_ATTACK) && !(pPlayer->v.oldbuttons & IN_ATTACK)
	   && ((strcmp (player_weapon, "flashbang.mdl") == 0)
		   || (strcmp (player_weapon, "hegrenade.mdl") == 0)
		   || (strcmp (player_weapon, "smokegrenade.mdl") == 0)))
	  DispatchSound ("weapons/pinpull.wav", GetGunPosition (pPlayer), 1.0, ATTN_NORM);
   return;
}

void PlayBulletSoundsForBots (edict_t *pPlayer)
{
   // this function is in charge of emulating the gunshot sounds for the bots. Since these sounds
   // are only predicted by the client, and bots have no client DLL, obviously we have to do the
   // work for them. We consider a client is told gunshot sound occurs when he receives the
   // msg_CurWeapon network message, which gets sent whenever a player is lowering his ammo.
   // That's why we hook those messages in MessageBegin(), and send the entity responsible of
   // it have a walk around here. Given the weapon this player is holding in his hand then, the
   // appropriate gunshot sound is played, amongst all the sounds that are listed in the
   // weaponsounds.cfg file. Then DispatchSound() is called to bring the selected sound to the
   // bot's ears.
   weapon_t *pPlayerWeapon;
   const char *texture_name;
   char texture_type;
   int player_index, sound_index;
   Vector v_gun_position;
   player_index = ENTINDEX (pPlayer) - 1; // get the player index
   if (!IsValidPlayer (pPlayer) || !players[player_index].is_alive)
	  return; // skip invalid and dead players
   if (DebugLevel.is_observer && !players[player_index].is_racc_bot)
	  return; // skip real players if in observer mode
   if (!(pPlayer->v.button & (IN_ATTACK | IN_ATTACK2)))
	  return; // cancel if player is not firing
   if (STRING (pPlayer->v.weaponmodel)[0] == 0)
	  return; // cancel if player has no weapon
   pPlayerWeapon = FindWeaponByModel (STRING (pPlayer->v.weaponmodel)); // get player's weapon
   if (pPlayerWeapon->id == 0)
	  return; // cancel if player has no weapon
   v_gun_position = GetGunPosition (pPlayer); // get this player's gun position
   // now select the sound according to rail (primary or secondary) and mode
   if (pPlayer->v.button & IN_ATTACK)
   {
	  // primary rail
	  if (pPlayer->v.weaponanim == 0)
		 DispatchSound (pPlayerWeapon->primary.sound1, v_gun_position, 1.0, ATTN_NORM);
	  else
		 DispatchSound (pPlayerWeapon->primary.sound2, v_gun_position, 1.0, ATTN_NORM);
   }
   else
   {
	  // secondary rail
	  if (pPlayer->v.weaponanim == 0)
		 DispatchSound (pPlayerWeapon->secondary.sound1, v_gun_position, 1.0, ATTN_NORM);
	  else
		 DispatchSound (pPlayerWeapon->secondary.sound2, v_gun_position, 1.0, ATTN_NORM);
   }
   // do we have to worry about ricochet sounds ?
   if (ricochetsound_count > 0)
   {
	  // did this player's last traceline hit something AND it is not a player ?
	  if ((players[player_index].tr.flFraction < 1.0) && !FNullEnt (players[player_index].tr.pHit)
		  && !(players[player_index].tr.pHit->v.flags & (FL_MONSTER | FL_CLIENT)))
	  {
		 // ask the engine for the texture name at the bullet hit point
		 texture_name = TRACE_TEXTURE (players[player_index].tr.pHit, v_gun_position, players[player_index].tr.vecEndPos);
		 // if the engine found the texture, ask the MOD DLL for the texture type
		 if (texture_name != NULL)
			texture_type = PM_FindTextureType ((char *) texture_name); // ask for texture type
		 // loop through all the ricochet sounds the bot knows until we find the right one
		 for (sound_index = 0; sound_index < ricochetsound_count; sound_index++)
		 {
			// is it this texture type the bullet just hit OR have we reached the default sound ?
			if ((texture_type == ricochetsounds[sound_index].texture_type)
				|| (ricochetsounds[sound_index].texture_type == '*'))
			   break; // then no need to search further
		 }
		 // bring this ricochet sound to the bots' ears
		 DispatchSound (ricochetsounds[sound_index].file_path, players[player_index].tr.vecEndPos, 0.9, ATTN_NORM);
	  }
   }
   return;
}

[Killaruna]

Hi all, it's been some time :-)

So here's my first question: What's the advantage of using DispatchSound() ? You'll have to hook to this function again to let the bots react to the sounds, and you'll need a lot of branching for the different cases there again. Why not do it in one go? Apart from that, nice code: I had the texture switch, too, but not the water sounds :-)

[Pierre-Marie Baty]

Haha, you forgot what were the engine functions don't you

DispatchSound is a function I wrote myself

I call this for every sound emitted in the game, to dispatch the sound with the right attenuation in one slot of the bot's ears, for the specified duration of the sound (yep, because in my bot, the sounds LAST in the bot's ears )

void DispatchSound (const char *sample, Vector v_origin, float volume, float attenuation)
{
   // this function brings the sound to the ears of the bots. Every time a sound is emitted in
   // the game somehow, this function has to be called. It cycles through all the bots that are
   // playing, and does the appropriate checks in order to determine if this bot will hear that
   // sound or not. In case it can, the function places the sound into the bot's ears structure.
   int bot_index;
   sound_t *sound;
   float attenuated_volume;
   if (sample == NULL)
	  return; // reliability check
   sound = FindSoundByFilename (sample); // find the sound we want in the global sound database
   if (sound->duration == 0)
	  return; // cancel if sound was not found
   // if debug mode is enabled, tell the user we are dispatching a sound around here
   if ((DebugLevel.ears > 2) || ((DebugLevel.ears > 1) && IsValidPlayer (pListenserverEntity) && ((v_origin - pListenserverEntity->v.origin).Length () <= 1500)))
	  printf ("DispatchSound() \"%s\" from (%.1f, %.1f, %.1f): vol %.1f, att %.1f\n", sound->file_path, v_origin.x, v_origin.y, v_origin.z, volume, attenuation);
   // cycle through all bot slots
   for (bot_index = 0; bot_index < RACC_MAX_CLIENTS; bot_index++)
   {
	  // is this slot used ?
	  if (bots[bot_index].is_active && IsValidPlayer (bots[bot_index].pEdict))
	  {
		 // is this sound NOT attenuated by distance ?
		 if (attenuation == ATTN_NONE)
			BotFeedEar (&bots[bot_index], sound, v_origin, volume); // if so, bot will hear it anyway
		 // else is that bot within the maximum hearing range of the PAS ?
		 // FIXME: ATM, I can't tell the difference between all the different attenuations !
		 else if ((v_origin - bots[bot_index].pEdict->v.origin).Length () < MAX_HEARING_DISTANCE)
		 {
			attenuated_volume = volume * ((MAX_HEARING_DISTANCE - (v_origin - bots[bot_index].pEdict->v.origin).Length ()) / MAX_HEARING_DISTANCE);
			BotFeedEar (&bots[bot_index], sound, v_origin, attenuated_volume); // bot hears attenuated sound
		 }
	  }
   }
   return; // done, sound dispatched to all bots in range
}

the BotFeedEar() thingy

void BotFeedEar (bot_t *pBot, sound_t *sound, Vector v_origin, float volume)
{
   // this function is in charge of finding, or freeing if necessary, a slot in the bot's ears
   // to put the sound pointed to by sound in. If no free slot is available, the sound that is
   // the most about to finish gets overwritten.
   static bot_ears_t *pBotEar;
   float nearest_fade_date;
   char i, selected_index;
   if (sound == NULL)
	  return; // reliability check
   pBotEar = &pBot->BotEars; // quick access to ear
   // find a free slot in bot's ears
   for (selected_index = 0; selected_index < BOT_EAR_SENSITIVITY; selected_index++)
	  if (pBotEar->noises[selected_index].fade_date < *server.time)
		 break; // break when a free slot is found
   // have we found NO free slot ?
   if (selected_index == BOT_EAR_SENSITIVITY)
   {
	  // no free slot found, so overwrite one, preferably the one most close to fade
	  // FIXME: wrong rule - given several sounds, WHICH ONE are we likely to ignore the most ?
	  nearest_fade_date = *server.time + 60.0;
	  selected_index = 0;
	  for (i = 0; i < BOT_EAR_SENSITIVITY; i++)
		 if (pBotEar->noises[i].fade_date < nearest_fade_date)
		 {
			nearest_fade_date = pBotEar->noises[i].fade_date;
			selected_index = i; // select the sound which is the most "finished"
		 }
   }
   // store the sound in that slot of the bot's ear
   pBotEar->noises[selected_index].file_path = sound->file_path; // link pointer to sound file path
   pBotEar->noises[selected_index].direction = BotEstimateDirection (pBot, v_origin); // remember origin
   pBotEar->noises[selected_index].fade_date = *server.time + sound->duration; // duration
   pBotEar->noises[selected_index].loudness = sound->loudness * volume; // loudness
   pBotEar->new_sound = TRUE; // notify the bot that it is hearing a new noise
   pBotEar->new_sound_index = selected_index; // mark new sound index for bot to check it
   // if debug mode is high, tell the developer that a new sound is coming to this bot
   if ((DebugLevel.ears > 1) && IsValidPlayer (pListenserverEntity) && ((pBot->pEdict->v.origin - pListenserverEntity->v.origin).Length () <= 100))
	  ServerConsole_printf ("NEW SOUND TO BOT %s's EAR: %s - LOUD %.1f - FAD %.1f\n",
							players[ENTINDEX (pBot->pEdict) - 1].connection_name,
							pBotEar->noises[selected_index].file_path,
							pBotEar->noises[selected_index].loudness,
							pBotEar->noises[selected_index].fade_date);
   return;
}

[Killaruna]

Uhm, yeah, I forgot the engine function names - partly ;-)
Now I see your system: Neat! - Although I doubt players will be able to appreciate these details... But knowing that you are programming the bot for your own pleasure: who cares, just do it! :-)

[TurtleRocker]

Don't forget to randomize the location of the sound - the farther away, the more random error. Otherwise, your bots will seem to be able to see through walls if they aim right for the sound source.

Human players, even with headphones, can't pinpoint the exact position of a sound, unless it is very close and they know the map layout very well.

[Pierre-Marie Baty]

No problem, Michael, this is exactly what I am doing:

pBotEar->noises[selected_index].direction = BotEstimateDirection (pBot, v_origin);

BotEstimateDirection translates a vector location in a general direction from the following ones:
- rather in front of the bot
- rather ahead on the left
- rather on the left
- rather behind on the left
- rather behind it
- rather behind on the right
- rather on the right
- rather in front on the right

// relative directions
#define DIRECTION_NONE 0
#define DIRECTION_FRONT (1 << 0)
#define DIRECTION_BACK (1 << 1)
#define DIRECTION_LEFT (1 << 2)
#define DIRECTION_RIGHT (1 << 3)

these are bitmasked

Distance is not taken in account, it's only the angle at which the sound comes to the bot. I believe it's more natural that way than to inflict (yet another) arbitrary randomization, what do you think ?

[TurtleRocker]

I think estimating the sounds relative position like you have done is a clever idea. It prevents "perfect hearing" and is computationally efficient.

However, I often store the estimated sound position and have the bot move there when hunting for enemies. If you only have a relative direction, you can't easily do that.

[Pierre-Marie Baty]

Right.
My bot's AI is not that far yet, hence I never faced really the problem. But it certainly will arise sooner or later.

I believe I can tackle it out easily, though... lemme see:

- The "direction" will encompass a certain group of walkfaces in the navmesh, which I can pop out quickly thanks to the topology hashtable
- The "distance" will be expressed in function of the sound's LOUDNESS, taking in account whether there is a direct LOS or not (a wall in between, for example).

Direction & distance => position.

I believe that's the most human-like way of doing things. However, this was just a 5 minute thought. I might be able to do better