It does matter. I am a stats teacher. You cannot justifiably compare numbers when their is too much variability between the two subjects. In order to make an accurate comparison one has to control all of the other variables. In order to accurately compare numbers one has to control ticket prices, stadium sizes, and number of games.
Example:
Sport A one year has an average seating of say 10,000 and all teams 10 teams sellout every game for 50 home games. That makes an average of 10,000 per game or a total of 5.0 fans.
Sport B one year has an average seating of say 50,000 and all 5 teams average 50% capacity for 10 home games. That makes and average of 25,000 per game or a total of 1.25 million fans.
I can then claim that Sport B surpasses Sport A in average attendance, even though they attracted 25% of the total fans of Sport A.
To accurately compare average fans per game we need to determine how many fans Sport A would attract if they too had average seating of 50,000 or Sport B had average seating of 10,000, and ticket prices are the same
To accurately compare total number of fans for a whole season we would need to compare the same total number of home games, the same total number of teams, and the same total average when seating capacity and ticket prices are held constant