If datetimes are sorted you can test if 2 consecutive values per groups are 0
:
m1 = df['score'].eq(0) & df.groupby('id')['score'].shift(-1).eq(0)
m2 = df['score'].eq(0) & df.groupby('id')['score'].shift().eq(0)
df = df[m1 | m2]
print (df)
date id name score
0 2020-10-19 1 Peter 0
3 2020-10-18 1 Peter 0
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…