2021年5月5日星期三

array memory issue in windows but not in Mac

I am working on a document classification problem.

I am getting memoryerror: unable to allocate 47.3 gib for an array with shape (4235, 3297459) and data type float 64 at this line of code:

x_train  = x_train .toarray()  

x_train contains docuument data

Here is a small snippet of my code.

vectorizer = TfidfVectorizer().fit(x_train )  x_train = vectorizer.transform(x_train )    x_train  = x_train .toarray()    model= Sequential()  model.add(Dense(units=30, kernel_initializer='uniform',activation='relu', input_dim=len(x_train[0]))  model.add(Dense(units=5, kernel_initializer='uniform', activation='softmax'))  model.fit(X_train, y_train, batch_size=30,epochs=50,class_weight=d_class_weights)  

When I am running this code on a 16 GB Macbook I am not getting any error but it is giving me this error on a 64GB Windows 10. May I know:

  1. why it is giving different result on different operating systems

  2. How can I resolve this error on Windows?

https://stackoverflow.com/questions/67411279/array-memory-issue-in-windows-but-not-in-mac May 06, 2021 at 11:05AM

没有评论:

发表评论