Multiplication of two huge dense matrices Hadamard-multiplied by a sparse matrix

I have two dense matrices A and B, and each of them has a size fo 3e5x100. Another sparse binary matrix, C, with size 3e5x3e5. I want to find the following quantity: C ∘ (AB'), where ∘ is Hadamard product (i.e., element wise) and B' is the transpose of B. Explicitly calculating AB' will ask for crazy amount of memory (~500GB). Since the end result won't need the whole AB', it is sufficient to only calculate the multiplication A_iB_j' where C_ij != 0, where A_i is the column i of matrix A and C_ij is the element at location (i,j) of the matrix C. A naive approach would be like the algorithm below:

result = numpy.initalize_sparse_matrix(shape = C.shape)  while True:   (i,j) = C_ij.pop_nonzero_index() #prototype function returns the nonzero index and then points to the next nonzero index   if (i,j) is empty:     break   result(i,j) = A_iB_j'  

This algorithm however takes too much time. Is there anyway to improve it using LAPACK/BLAS algorithms? I am coding in Python so I think numpy can be more human friendly wrapper for LAPACK/BLAS.

https://stackoverflow.com/questions/65802720/multiplication-of-two-huge-dense-matrices-hadamard-multiplied-by-a-sparse-matrix January 20, 2021 at 11:36AM