Wednesday, March 08, 2006

st: memory required for -merge-

Hi all,

Is there a rule for the amount of memory required for -merge-? I keep getting a "no room to add more variables due to width" error on the same merge. I am attempting to -merge- on only one variable (placefip), and 67% of memory is free before I attempt the -merge- (i.e., while the "master" dataset is loaded). The "using dataset is small, and I've got the memory set quite high (900m). So, I can't understand why I am running into memory problems.

This is the info on the "master" dataset:

. des

Contains data from ../data/geog/geog_pooled.dta obs: 20,873,141 vars: 4 8 Mar 2006 21:32 size: 333,970,256 (66.5% of memory free) (_dta has notes) ------------------------------------------------------------------------ ------- storage display value variable name type format label variable label ------------------------------------------------------------------------ ------- cityres long %05.0f * City of residence msarfip int %04.0f * PMSA/MSA of residence (FIPS) year int %4.0g placefip long %07.0f * Place (city) of residence (FIPS) * indicated variables have notes ------------------------------------------------------------------------ -------

And this is the info on the "using" dataset:

. des using ../data/geog/geog_cc

Contains data obs: 25,150 8 Mar 2006 22:21 vars: 3 size: 1,207,200 ------------------------------------------------------------------------ ------- storage display value variable name type format label variable label ------------------------------------------------------------------------ ------- macci byte %1.0g Center city placefip long %07.0f place_name str39 %39s ------------------------------------------------------------------------ ------- Sorted by: placefip

This is the memory usage I see after the error, when the "master" dataset is loaded:

. memory bytes -------------------------------------------------------------------- Details of set memory usage overhead (pointers) 83,492,564 8.38% data 250,477,692 25.14% ---------------------------- data + overhead 333,970,256 33.53% free 662,176,944 66.47% ---------------------------- Total allocated 996,147,200 100.00% -------------------------------------------------------------------- Other memory usage set maxvar usage 1,848,666 set matsize usage 1,315,200 programs, saved results, etc. 41,128 --------------- Total 3,204,994 ------------------------------------------------------- Grand total 999,352,194

Any ideas?

-- Danielle H Ferry

* * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/


Tag:


Links to this post:

Create a Link



<< Home

This page is powered by Blogger. Isn't yours?