python 3.x - Beautiful Soup Nested Loops

Question

I was hoping to create a list of all of the firms featured on this list. I was hoping each winner would be their own section in the HTML but it looks like there are multiple grouped together across several divs. How would you recommend going about solving this? I was able to pull all of the divs but i dont know how to cycle through them appropriately. Thanks!

import requests
from bs4 import BeautifulSoup
import csv

request = requests.get("https://growthcapadvisory.com/growthcaps-top-40-under-40-growth-investors-of-2020/")
text = request.text

soup = BeautifulSoup(text, 'html.parser')
element = soup.find()

person = soup.find_all('div', class_="under40")

question from:https://stackoverflow.com/questions/65622852/beautiful-soup-nested-loops

Answer 1

This solution uses css selectors

import requests
from bs4 import BeautifulSoup

request = requests.get("https://growthcapadvisory.com/growthcaps-top-40-under-40-growth-investors-of-2020/")
text = request.text

soup = BeautifulSoup(text, 'html.parser')
# if you have an older version you'll need to use contains instead of -soup-contains
firm_tags = soup.select('h5:-soup-contains("Firm")  strong')
# extract the text from the selected bs4.Tags
firms = [tag.text for tag in firm_tags]
# if there is extra whitespace
clean_firms = [f.strip() for f in firms]

It works by selecting all the strong tags whose parent h5 tag contain the word "Firm"

See the SoupSieve Docs for more info on bs4's CSS Selectors