Monday, March 20, 2006

RE: st: ivreset

Hi all. I think I spoke too soon. The differences between the different approaches to rescaling are driven, I think, by numerical precision problems and collinearities.

Here are the 3 versions of the basic ovtest using squares and cubes: no rescaling, rescaling yhat and then squaring and cubing, and squaring and cubing followed by rescaling. This uses the code that I posted to the list earlier today. All three generate the same results. When a yhat^4 is included, as in the example I posted earlier, is when collinearities arise and different test statistics can be generated.

--Mark

. * Squared and cubed . * Unrescaled RESET . qui regress lw s yhat2 yhat3

. testparm yhat*

( 1) yhat2 = 0 ( 2) yhat3 = 0

F( 2, 754) = 0.87 Prob > F = 0.4188

. * yhat that is first ^2, ^3, then rescaled . * Output identical to unrescaled RESET . qui regress lw s yhatrr2 yhatrr3

. testparm yhat*

( 1) yhatrr2 = 0 ( 2) yhatrr3 = 0

F( 2, 754) = 0.87 Prob > F = 0.4188

. * yhat that is first rescaled, then ^2, ^3 . * Output different from unrescaled RESET . qui regress lw s yhatr2 yhatr3

. testparm yhat*

( 1) yhatr2 = 0 ( 2) yhatr3 = 0

F( 2, 754) = 0.87 Prob > F = 0.4188

> -----Original Message----- > From: owner-statalist@hsphsun2.harvard.edu > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of > Schaffer, Mark E > Sent: 20 March 2006 16:57 > To: statalist@hsphsun2.harvard.edu > Subject: RE: st: ivreset > > Arne, > > I think you are onto something here. The rescaling should > perhaps take place *after* creating the polynomials, not before. > > That said, Stata's official -ovtest- also rescales first, > then creates the polynomials. When I wrote -ivreset-, I had > used replication of the output of -ovtest- as a check, and I > used the same approach to rescaling yhat, namely rescale and > then create polynomials (rather than create polynomials and > then rescale). > > Here is an example that demonstrates that (a) rescaling after > creating the polynomials leaves the reset statistic > unchanged, and (b) rescaling before creating the polynomials > replicates the output of official -ovtest-. Note that there > are collinearity problems with (a) even after rescaling, > which makes me hesitate... > > Does anyone else want to comment? > > --Mark > > *********** do code *************** > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta, > clear capture drop yhat* qui regress lw s qui predict double yhat > * yhatr=rescaled yhat > sum yhat, meanonly > qui gen double yhatr = (yhat-r(min))/(r(max)-r(min)) qui gen > double yhat2=yhat^2 qui gen double yhat3=yhat^3 qui gen > double yhat4=yhat^4 qui gen double yhatr2=yhatr^2 qui gen > double yhatr3=yhatr^3 qui gen double yhatr4=yhatr^4 > * yhatrr2=rescaled yhat2 > * yhatrr3=rescaled yhat3 > * yhatrr4=rescaled yhat4 > sum yhat2, meanonly > qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min)) sum > yhat3, meanonly qui gen double yhatrr3 = > (yhat3-r(min))/(r(max)-r(min)) sum yhat4, meanonly qui gen > double yhatrr4 = (yhat4-r(min))/(r(max)-r(min)) sum yhat* > > * Unrescaled RESET > qui regress lw s yhat2 yhat3 yhat4 > testparm yhat* > * yhat that is first ^2, ^3, ^4, then rescaled > * Output identical to unrescaled RESET > qui regress lw s yhatrr2 yhatrr3 yhatrr4 testparm yhat* > * yhat that is first rescaled, then ^2, ^3, ^4 > * Output different from unrescaled RESET qui regress lw s > yhatr2 yhatr3 yhatr4 testparm yhat* > * Stata's built-in ovtest > * Output again different from unrescaled RESET qui regress lw s Ovtest > > ************* output **************** > > > . use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta, > clear (Wages of Very Young Men, Zvi Griliches, J.Pol.Ec. 1976) > > . capture drop yhat* > > . qui regress lw s > > . qui predict double yhat > > . * yhatr=rescaled yhat > . sum yhat, meanonly > > . qui gen double yhatr = (yhat-r(min))/(r(max)-r(min)) > > . qui gen double yhat2=yhat^2 > > . qui gen double yhat3=yhat^3 > > . qui gen double yhat4=yhat^4 > > . qui gen double yhatr2=yhatr^2 > > . qui gen double yhatr3=yhatr^3 > > . qui gen double yhatr4=yhatr^4 > > . * yhatrr2=rescaled yhat2 > . * yhatrr3=rescaled yhat3 > . * yhatrr4=rescaled yhat4 > . sum yhat2, meanonly > > . qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min)) > > . sum yhat3, meanonly > > . qui gen double yhatrr3 = (yhat3-r(min))/(r(max)-r(min)) > > . sum yhat4, meanonly > > . qui gen double yhatrr4 = (yhat4-r(min))/(r(max)-r(min)) > > . sum yhat* > > Variable | Obs Mean Std. Dev. Min Max > -------------+-------------------------------------------------------- > yhat | 758 5.686739 .2156493 5.261107 6.130727 > yhatr | 758 .4894459 .2479809 0 1 > yhat2 | 758 32.38544 2.473995 27.67924 37.58581 > yhat3 | 758 184.7003 21.3139 145.6234 230.4284 > yhat4 | 758 1054.929 163.4247 766.1405 1412.693 > -------------+-------------------------------------------------------- > yhatr2 | 758 .3009707 .2769761 0 1 > yhatr3 | 758 .2142687 .2681644 0 1 > yhatr4 | 758 .1671979 .2530434 0 1 > yhatrr2 | 758 .4750582 .2497327 0 1 > yhatrr3 | 758 .4607848 .2513286 0 1 > -------------+-------------------------------------------------------- > yhatrr4 | 758 .4466593 .252763 0 1 > > . > . * Unrescaled RESET > . qui regress lw s yhat2 yhat3 yhat4 > > . testparm yhat* > > ( 1) yhat2 = 0 > ( 2) yhat3 = 0 > ( 3) yhat4 = 0 > Constraint 1 dropped > > F( 2, 754) = 0.87 > Prob > F = 0.4191 > > . * yhat that is first ^2, ^3, ^4, then rescaled . * Output > identical to unrescaled RESET . qui regress lw s yhatrr2 > yhatrr3 yhatrr4 > > . testparm yhat* > > ( 1) yhatrr2 = 0 > ( 2) yhatrr3 = 0 > ( 3) yhatrr4 = 0 > Constraint 1 dropped > > F( 2, 754) = 0.87 > Prob > F = 0.4191 > > . * yhat that is first rescaled, then ^2, ^3, ^4 . * Output > different from unrescaled RESET . qui regress lw s yhatr2 > yhatr3 yhatr4 > > . testparm yhat* > > ( 1) yhatr2 = 0 > ( 2) yhatr3 = 0 > ( 3) yhatr4 = 0 > > F( 3, 753) = 0.59 > Prob > F = 0.6216 > > . * Stata's built-in ovtest > . * Output again different from unrescaled RESET . qui regress lw s > > . ovtest > > Ramsey RESET test using powers of the fitted values of lw > Ho: model has no omitted variables > F(3, 753) = 0.59 > Prob > F = 0.6216 > > . > end of do-file > > ************************************* > > > -----Original Message----- > > From: owner-statalist@hsphsun2.harvard.edu > > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of > Arne Risa > > Hole > > Sent: 20 March 2006 16:03 > > To: statalist@hsphsun2.harvard.edu > > Subject: Re: st: ivreset > > > > Hi Mark, > > > > Thank you very much for the clarifications. The rescaling > does have an > > impact on the test statistic, however; this can be seen from using > > Austin's code and comparing the results with and without the line: > > > > replace yhat = (yhat-r(min))/(r(max)-r(min)) > > > > So even when the correct "optimal" forecast of yhat is used > > (yhat=X-hat*beta-hat), rescaling the forecast affects the > result. This > > is not a problem in the case of -ovtest-, however, since the Reset > > test statistic is invariant to the rescaling in the OLS case. > > > > Sorry for going on about this, but it seems to me that > since the two > > statistics differ the correct statistic is the one without the > > rescaling of the (yhat=X-hat*beta-hat) forecast (even > though this may > > introduce numerical precision problems in some cases). > > > > Cheers, > > Arne > > > > On 20/03/06, Schaffer, Mark E <M.E.Schaffer@hw.ac.uk> wrote: > > > Arne, > > > > > > > -----Original Message----- > > > > From: owner-statalist@hsphsun2.harvard.edu > > > > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf > > Of Arne Risa > > > > Hole > > > > Sent: 18 March 2006 11:43 > > > > To: statalist@hsphsun2.harvard.edu > > > > Subject: RE: st: ivreset > > > > > > > > Austin, Mark, > > > > > > > > Thank you both for your replies, rescaling the forecast did the > > > > trick (sorry for the bad formatting of my code before, > it looked > > > > fine in my email programme). > > > > > > > > I understand the motivation behind the rescaling, but I'm > > slightly > > > > concerned about the fact that it produces a different > > test statistic > > > > compared to using the actual forecast. Note that this > is not the > > > > case when using the Reset test following OLS - the test > > statistic is > > > > invariant to the rescaling in this case. > > > > > > > > I would think that since the two approaches (rescaling/ no > > > > rescaling) produce different results, the correct test > > statistic is > > > > that using the actual forecast? > > > > > > There are two different issues here. Rescaling is one, and > > it is, in > > > some sense, a side issue. The problem is that sometimes > > the yhat has > > > large-ish values, and the higher order polynomials of > yhat that are > > > included in the artificial regression can get so big that the > > > regression fails for numerical precision reasons. Stata's > > own version > > > of the reset test, -ovtest-, also does this rescaling. The test > > > statistic is, of course, invariant in theory to the units > > used and hence to rescaling. > > > > > > The other issue is the one you might have missed. As > > Austin pointed > > > out, the IV version of the RESET test cannot use standard fitted > > > values that would be generated by -predict- after estimation using > > > -ivreg- or -ivreg2-. These are yhat=X*beta-hat, and the > problem is > > > that X includes some endogenous regressors. > > > > > > As the help file for -ivreset- explains, there are two > alternatives. > > > One is to use reduced form predictions for yhat, i.e., > regress y on > > > all the exogenous variables (including the excluded > > instruments) and > > > then use -predict-. The other is to get what Pesaran and > > Taylor call > > > the "optimal forecast" of yhat. This is not yhat=X*beta-hat, but > > > yhat=X-hat*beta-hat, where X-hat includes the reduced form > > predicted > > > values of the endogenous regressors (rather than the > actual values). > > > The code that Austin kindly posted to Statalast implemented > > the latter. > > > > > > Cheers, > > > Mark > > > > > > > Cheers > > > > Arne > > > > > > > > On Mar 17 2006, Schaffer, Mark E wrote: > > > > > > > > > Austin, Arne, > > > > > > > > > > > -----Original Message----- > > > > > > From: Austin Nichols [mailto:austinnichols@gmail.com] > > > > > > Sent: 16 March 2006 23:30 > > > > > > To: statalist@hsphsun2.harvard.edu > > > > > > Subject: Re: st: ivreset > > > > > > > > > > > > -findit ivreset- then -help ivreset- when installed has a > > > > excellent > > > > > > exposition that begins: > > > > > > > > > > > > As Pagan and Hall (1983) and Pesaran and Taylor (1999) > > > > point out, a > > > > > > RESET test for an IV regression cannot use the standard > > > > IV predicted > > > > > > values X*beta-hat, because X includes endogenous > > > > regressors that are > > > > > > correlated with u. > > > > > > > > > > > > Try this code instead: > > > > > > > > > > > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta > > > > > > qui ivreg2 lw s expr tenure rns smsa (iq=med kww) predict > > > > ytilde mat > > > > > > b=e(b) mat li b qui regress iq s expr tenure rns > smsa med kww > > > > > > qui predict double xh gen yhat=ytil-b[1,1]*iq+b[1,1]*xh sum > > > > > > yhat, meanonly qui replace yhat = > > (yhat-r(min))/(r(max)-r(min)) > > > > > > qui gen double yhat2=yhat^2 qui ivreg2 lw s expr > > tenure rns smsa > > > > > > yhat2 (iq=med kww) test yhat2 qui ivreg2 lw s expr > tenure rns > > > > smsa (iq=med > > > > > > kww) ivreset > > > > > > > > > > > > Now, as to why > > > > > > replace yhat = (yhat-r(min))/(r(max)-r(min)) I cannot > > > > tell you, but > > > > > > it's in ivreset.ado > > > > > > > > > > It's to rescale yhat so that when it's squared, > cubed, etc., it > > > > > doesn't get wildly out of scale relative to the other > > regressors. > > > > > This can cause problems for the regression that includes > > > > these terms. > > > > > > > > > > Cheers, > > > > > Mark > > > > > > > > > > > > > > > > On 16 Mar 2006 19:10:13 +0000, Arne Risa Hole wrote: > > > > > > > I am using ivreset to do a Pesaran-Taylor Reset test > > > > after ivreg2. > > > > > > > However, I am not able to replicate the result > from ivreset > > > > > > manually. For example: > > > > > > > > > > > > > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta > > > > > > > > > > > > > > > > > > > > > > * > > > > > * For searches and help try: > > > > > * http://www.stata.com/support/faqs/res/findit.html > > > > > * http://www.stata.com/support/statalist/faq > > > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > > > > > > > > * > > > > * For searches and help try: > > > > * http://www.stata.com/support/faqs/res/findit.html > > > > * http://www.stata.com/support/statalist/faq > > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > > > > > > > > > * > > > * For searches and help try: > > > * http://www.stata.com/support/faqs/res/findit.html > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > > * > > * For searches and help try: > > * http://www.stata.com/support/faqs/res/findit.html > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > * > * For searches and help try: > * http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > >

* * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/


Tag:


Links to this post:

Create a Link



<< Home

This page is powered by Blogger. Isn't yours?