python - How to set activation or optimiser for deeplearning model only accept 1,0 for RNN

Question

For example

I have time seriese datalike this

[[1,0,0,0] [1,0,0,1],[1,0,1,0],[1,1,0,0]],,,,

and it predict the next one from past two.

I want to put [[1,0,0,0],[1,0,0,1]] and get [1,0,1,0]

So I made model like these below.

input_len = 2
n_in = 4
n_hidden = 512
model = Sequential()

model.add(LSTM(n_hidden, input_shape=(input_len,n_in), return_sequences=True))
        
model.add(Dropout(0.1))
model.add(LSTM(n_hidden,  return_sequences=False))
        
model.add(Dense(n_hidden, activation="linear")) 
        
model.add(Dense(n_in, activation="linear"))
opt = Adam(lr=0.001)
model.compile(loss='mse', optimizer=opt)
model.summary()

#trainning and validate data 

X     #X.shape (800, 2, 4) [ [[1,0,0,1],[1,0,0,1]],[[1,0,0,1],[1,0,0,0]],,,
Y     #Y.shape (200, 2, 4)
val_x #val_x.shape (800,1,4) [[1,0,1,0]][1,1,1,0],,,,
val_y #val_y.shape (200,1,4)

history = model.fit(x, y, epochs=50,validation_data=(val_x, val_y))

#then predict
in_ = np.array[[1,0,0,1][1,1,1,1]]
out_ = model.predict(in_)
print(out_)

I expect as the result at least 1 or 0.

however I get the number like this [[4.9627638e-01 1.4797167e-01 3.3314908e-01 1.3892795e-04]]

I guess this is relevant with activation or optimizer...

Am I correct? or how should I do for 1 and 0 data?

---

change linear to relu

the result becomes between [0.41842282 0.1275532 0. 0.4288069]

However still it is not 0 or 1....

question from:https://stackoverflow.com/questions/65661095/how-to-set-activation-or-optimiser-for-deeplearning-model-only-accept-1-0-for-rn

Answer 1

Model output can not be discrete because it should be differentiable. Try to add something like that:

out_ = tf.cast(tf.math.greater(out_, 0.5), tf.int32)

It is not right prediction, but the accuracy depends on your data (e.g. if your data is random and there is no pattern - then you get 6% accuracy). Try to train based on only [[1,0,0,0] [1,0,0,1],[1,0,1,0]] to be sure that your model works.