replaced http://mathematica.stackexchange.com/ with https://mathematica.stackexchange.com/

Source Link

edited Apr 13, 2017 at 12:55

1

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. If you look at my question Advanced TuplingAdvanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

Let's use that question to illustrate. Here are the data:

a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 22, 23, 25, 26, 28}; 
b = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 30, 31, 33, 37, 41}; 
c = {6, 10, 13, 14, 15, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 35, 37, 39};  
d = {17, 19, 25, 30, 31, 33, 34, 35, 36, 38, 44}; 
e = {31, 41, 45, 47}; 
f = {23, 26, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53};

And here is one of the solutions posted in that question:

newTuples[t_, x_] := 
  Flatten[ ParallelTable[Append[s, #] & /@ Complement[x, s], {s, t}], 1];

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally days to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how Mathematica was intended to work. I must be missing something.

What are some memory-efficient ways to visualize a data set as large as the output of the above function?

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

Let's use that question to illustrate. Here are the data:

a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 22, 23, 25, 26, 28}; 
b = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 30, 31, 33, 37, 41}; 
c = {6, 10, 13, 14, 15, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 35, 37, 39};  
d = {17, 19, 25, 30, 31, 33, 34, 35, 36, 38, 44}; 
e = {31, 41, 45, 47}; 
f = {23, 26, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53};

And here is one of the solutions posted in that question:

newTuples[t_, x_] := 
  Flatten[ ParallelTable[Append[s, #] & /@ Complement[x, s], {s, t}], 1];

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally days to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how Mathematica was intended to work. I must be missing something.

What are some memory-efficient ways to visualize a data set as large as the output of the above function?

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

Let's use that question to illustrate. Here are the data:

a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 22, 23, 25, 26, 28}; 
b = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 30, 31, 33, 37, 41}; 
c = {6, 10, 13, 14, 15, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 35, 37, 39};  
d = {17, 19, 25, 30, 31, 33, 34, 35, 36, 38, 44}; 
e = {31, 41, 45, 47}; 
f = {23, 26, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53};

And here is one of the solutions posted in that question:

newTuples[t_, x_] := 
  Flatten[ ParallelTable[Append[s, #] & /@ Complement[x, s], {s, t}], 1];

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally days to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how Mathematica was intended to work. I must be missing something.

What are some memory-efficient ways to visualize a data set as large as the output of the above function?

added content from OP's comments and made title more explicit

Source Link

edited Mar 6, 2012 at 22:00

Verbeia

34.4k
9
109
225

See Spot Run... and Run... and Run Efficiently Visualising Very Large Data Sets (without running out of memory)

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. So what good is it? If If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

Let's use that question to illustrate. Here are the data:

a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 22, 23, 25, 26, 28}; 
b = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 30, 31, 33, 37, 41}; 
c = {6, 10, 13, 14, 15, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 35, 37, 39};  
d = {17, 19, 25, 30, 31, 33, 34, 35, 36, 38, 44}; 
e = {31, 41, 45, 47}; 
f = {23, 26, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53};

And here is one of the solutions posted in that question:

newTuples[t_, x_] := 
  Flatten[ ParallelTable[Append[s, #] & /@ Complement[x, s], {s, t}], 1];

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally DAYSdays to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how M!Mathematica was intended to work. I must be missing something?.

What are some memory-efficient ways to visualize a data set as large as the output of the above function?

See Spot Run... and Run... and Run

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. So what good is it? If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally DAYS to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how M! was intended to work. I must be missing something?

Efficiently Visualising Very Large Data Sets (without running out of memory)

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

Let's use that question to illustrate. Here are the data:

a = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 18, 21, 22, 23, 25, 26, 28}; 
b = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 27, 28, 30, 31, 33, 37, 41}; 
c = {6, 10, 13, 14, 15, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 32, 35, 37, 39};  
d = {17, 19, 25, 30, 31, 33, 34, 35, 36, 38, 44}; 
e = {31, 41, 45, 47}; 
f = {23, 26, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53};

And here is one of the solutions posted in that question:

newTuples[t_, x_] := 
  Flatten[ ParallelTable[Append[s, #] & /@ Complement[x, s], {s, t}], 1];

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally days to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how Mathematica was intended to work. I must be missing something.

What are some memory-efficient ways to visualize a data set as large as the output of the above function?

added link to related question

Source Link

edited Mar 6, 2012 at 15:34

rcollyer

34.1k
7
93
192

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. So what good is it? If you look at my question 'Advanced Tupling'Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally DAYS to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how M! was intended to work. I must be missing something?

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. So what good is it? If you look at my question 'Advanced Tupling' there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally DAYS to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how M! was intended to work. I must be missing something?

I have put a few really hard problems in combinatorics up against Mathematica 8. I'd have to say that it works really well, until you want to view the data. So what good is it? If you look at my question Advanced Tupling there is an elegant solution. One line function. Data of about 12 million sets given the sample data presented.

One would however like to verify the results. From everything seen, it looks good. But one wants to see it all. Click 'Show All' output and the program grabs all available memory resources, including consuming all available disk space for page file and still takes literally DAYS to complete. Is there an efficient way to render data?

I have had the same problem with graphical plots, it just chokes up after processing. It is as if the entire problem has to be reprocessed for it to render. Surely this cannot be how M! was intended to work. I must be missing something?