Skip to content

MemoryError #33

@ereyred

Description

@ereyred

Hello, I'm running very large samples with lots of genes (> 30,000) and before it gave me "Job killed" but then I moved to a bigger cluster and it gives me:
Maximum number of Gibbs sampling iterations: 1000; terminating Gibbs sampling now.
Optimizing parameters for optimal clusters.
Cluster 1, 3557 genes
Traceback (most recent call last):
File "DP_GP_cluster.py", line 623, in
iter_num_at_birth=iter_num)
File "DP_GP/core.pyx", line 233, in DP_GP.core.dp_cluster.init (DP_GP/core.c:6979)
File "/py27/lib/python2.7/site-packages/GPy/core/parameterization/parameterized.py", line 27, in call
self.parameters_changed()
File "/py27/lib/python2.7/site-packages/GPy/core/gp.py", line 188, in parameters_changed
self.posterior, self._log_marginal_likelihood, self.grad_dict = self.inference_method.inference(self.kern, self.X, self.likelihood, self.Y_normalized, self.mean_function, self.Y_metadata)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/inference/latent_function_inference/exact_gaussian_inference.py", line 52, in inference
K = kern.K(X)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/kernel_slice_operations.py", line 79, in wrap
ret = f(self, s.X, s.X2, *a, **kw)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 184, in call
return cacher(*args, **kwargs)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 120, in call
self.add_to_cache(cache_id, inputs, self.operation(*args, **kw))
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/add.py", line 39, in K
return reduce(np.add, (p.K(X, X2) for p in which_parts))
MemoryError

Do I need more CPUs or something? Or is it really not possible to cluster samples with so many genes?
Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions