You will find the corresponding option inside:
Dashboard >> Appearance >> Customize >> BuddyPress Nouveau >> General BP Setting.
You will get an option to Configure a general BuddyPress appearance, i.e., you can use the round style for member and group avatars.