python pandas dataframe mutate all elements as (element / max element in row)

Question

df=

x1, x2, x3, x4, x5...x90, 'ignore'
37.4, 35.6, 43.5, 33.3, 42.4...42.5, 'ignoe'
32.2, 37.2, 43.4, 34.3, 82.4...22.1, 44
33.5, 45.5, 25.9, 23.8, 24.2...52.2, True
35.4, 30.6, 33.3, 33.3, 42.5...82.5, 'ignre'

I want to mutate all of the elements so they will all be between 0-1, by dividing each element by the max element in the row, ignoring the last column (there will not be any negative numbers or 0)

The problem I am having is it needs to be performant on millions of rows but iterating over rows in pandas is very slow

I believe this works but it is very slow

for index, row in df.iterrows():
    rowMax = max(row.drop(columns='ignore'))
    for i in range(1, 91):
        df.loc[index, 'x' + str(i)] = df.loc[index, 'x' + str(i)] / rowMax

output:

x1, x2, x3, x4, x5...x90, 'ignore'
0.86, 0.82, 1.00, 0.77, 0.97...0.98, 'ignoe'

question from:https://stackoverflow.com/questions/65878816/python-pandas-dataframe-mutate-all-elements-as-element-max-element-in-row

Answer 1

df.update(df.iloc[:,0:90].div(df.max(axis=1), axis=0))

First, you should grab the part of the dataframe you want to do the divide on: df.iloc[:,0:90] which means for every row pick the columns from 0 to 89.

Then, you grab the max of each row with: df.max(axis=1). Because your ignore column does not have all values as numerical, it would be ignored by max function so no worries about that.

Then, you do a divide with axis=0 meaning that divide on each row.

And, as the last step, you update the dataframe.