Re: [Help-glpk] Writing Output File to MS Access

[Meketon, Marc]

One possibility to get around the memory problem is to use a "for" loop and a "print" statement that outputs to a text file, and then use a macro in Access to import the text file.

 

I think the usage is something like (I don't have the documentation in front of me) for creating a CSV file

 

for {pl in PLANTS, ma in MACHINES, dc in WAREHOUSES, pm in PRODUCTS:flow_pl_dc[pl,ma,dc,pm]>0}

print "%s,%s,%s,%s,%g\n",        pl, ma, dc, pm, flow_pl_dc[pl,ma,dc,pm] >> SOME_CSV_FILE.CSV;

---

From: help-glpk-bounces+address@hidden [mailto:help-glpk-bounces+address@hidden On Behalf Of Alex Morelli
Sent: Friday, September 02, 2011 2:47 AM
To: address@hidden
Subject: [Help-glpk] Writing Output File to MS Access

Hello,

I have a large MIP model (lots of data) for a location - allocation problem.  The model fails due to lack of memory, so I have reduced the data set to validate the model.

I am using a Windows XP 32 bit with 4GB ram (usable is <3GB after system resources - as I understand this stuff).  I will be acquiring a 64bit workstation with lots more memory.

I have used the "build MPS" option in Gusek, creating a 28Mb file - if that provides any indication of the size of the model.

Since having trouble some time ago reading input data from MS Access, I have used the update provided by Xypron successfully (thanks very much and my apologies for not having thanked you earlier).

Now, however, writing the output causes a lack of memory error and the model fails even with the reduced data set.  I have been watching the memory monitor through Windows Task Manager and note that the reduced data set finds an integer (they are all binary) optimal solution after about 2500 secs at which time the reported available memory is c.c. 1.9GB.

When I include the Access table output the memory reduces rapidly and the program fails.  The following is one of the Access output table syntax (which I assume is correct, because the smaller output tables update correctly).

table result_flow_pl_dc {pl in PLANTS, ma in MACHINES, dc in WAREHOUSES, pm in PRODUCTS:flow_pl_dc[pl,ma,dc,pm]>0} OUT "ODBC"

        'DSN=glpk_Yorke'
        'DELETE FROM glpk_flow_pl_dc;'
        'INSERT INTO glpk_flow_pl_dc VALUES (?,?,?,?,?)':
        pl~PL, ma~MACHINE, dc~DC, pm~MATERIAL, flow_pl_dc[pl,ma,dc,pm]~QTY;

I have checked and confirmed that the Access table has the same field names and the following is the table structure / data types:

PL                      Number - Long Integer (which is the same as the input data
MACHINE         Number - Long Integer (which is the same as the input data
DC                      Number - Long Integer (which is the same as the input data
MATERIAL                Text - 50 (the input data table has 255, but the actual maximum length is 18).
QTY                     Number - Double (same as the input table)

Is this problem expected or is there something that I can do to get it to complete?

The reason that I am doing this is that the output file that I get from --output is very large, with many zeros.  I adopted this approach after reading some of the help archives.  If there are better / preferred methods, I would be grateful for any advice / suggestions.

(I have tried the csv output with the same outcome).
Regards, Alex

The information included in this email and any attached documents from the Sender may be confidential and/or subject to legal privilege. This information is intended to be for the use of the addressee only. The information contained in this email may not represent
 the views or opinions of the Sender’s employer. If you are not the intended recipient, note that any disclosure, photocopying, distribution or use of the contents of the emailed information is prohibited. If you have received this email in error, please contact
 the sender promptly and delete the email from any system and destroy any hard copies. Thank you.

---

This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies,
 and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation.