Better Than The Models ?

[Pie Man]

You're all completely missing my point.

I'm not interested in criticising anyone's forecast, and I'm not going to make any forecasts myself. I want to engage in a proper scientific debate, which has to include an airing and discussion of the forecasting methods being used, not just the forecast outcomes.

'Forecasters' who will not discuss their methods have little credibility to me - regardless of the outcome of the forecast.

Anyone can roll a dice.

Pieman

No... next time we'll have a control with one of MB's critics, Pieman, you, Steve Murr, independently come up with their own specific forecast. We'll see who's closest.

If it is all down to random chance, as you seem to think, you are all just as likely to get it right as MB. Or even more right.

Ed: MB will pick the date and will ask us for our forecasts before he posts his. I may even have a go myself

PaulB - I wholeheartedly agree with you. Your post is a breath of fresh air compared with those of many other contributors.

Pieman

My first (and probably my only) post on this thread.

Credit could, and can be given to some extent, if Nick was forecasting a general pattern. However, Nick has given a very specific forecast with specific pressure readings for specific locations. Now, no disrepect to Nick here, but anyone who knows a thing or two about meterology will acknowledge the fact that it is impossible to come up with detailed pressure charts two months or so before a specific day like Nick has and expect to be correct. Whilst the general pattern may be right, Nick's forecast will have to go down as being wrong purely because he is being so specific and he has indicated the importance of this accuracy himself. We can already see that the chart Nick came up with for yesterday was technically wrong, although the overall pattern wasn't too far out. A 1005mb low over Spain where a 1030mb high actually was forecast to be can not go down as being right. At the same time, a 1025mb ridge of high pressure over Ireland is not the same as a 1040mb high sitting there.

I don't begrudge anyone trying to come up with new methods of long range forecasting, but I do feel very strongly that it is a case of trying to run before you can even walk. Drawing pressure maps for 2 months out with self proclaimed high confidence is never going to work, whatever the methodlogy. However, going on and drawing a sequence of pressure maps for a specific time period 2 months out is just a waste of time to be frank, let alone going on to predict an event that is actually probably scientifically impossible to actually occur! No disrespect to you Nick, but it will never work. Stick to general patterns and give yourself some leeway and you may be onto something, but don't get into specifics because you will be fighting a losing battle from day 1.

Edited January 29, 2011 by Pie Man

[AtlanticFlamethrower]

You're all completely missing my point.

I'm not interested in criticising anyone's forecast, and I'm not going to make any forecasts myself. I want to engage in a proper scientific debate, which has to include an airing and discussion of the forecasting methods being used, not just the forecast outcomes.

You need to start a new thread for this.

'Forecasters' who will not discuss their methods have little credibility to me - regardless of the outcome of the forecast.

Anyone can roll a dice.

Pieman

Your point has been made. Can this be the last time?

[Pie Man]

Quote

'Forecasters' who will not discuss their methods have little credibility to me - regardless of the outcome of the forecast.

Anyone can roll a dice.

Pieman

Your point has been made. Can this be the last time?

Yes it can.

Scientia non habet inimicum nisp ignorantem

Pieman

[pottyprof]

Yes it can.

Scientia non habet inimicum nisp ignorantem

Pieman

But you are being ignorant... How many more ways do you want to be told "what you see is all you are getting"? You cannot demand from someone, what is possibly their own intellectual property. I'm sure that once the method has been refined, the method will be made public.

I notice it is fine for climate scientists to hide behind a wall of vagueness with their methods at times...

[peterf]

Yes it can.

Scientia non habet inimicum nisp ignorantem

Pieman

if you dont agree or like the thread.why not just go away? i could phrase it more direct but the swear filter may explode.

[SMU]

My first (and probably my only) post on this thread.

Credit could, and can be given to some extent, if Nick was forecasting a general pattern. However, Nick has given a very specific forecast with specific pressure readings for specific locations. Now, no disrepect to Nick here, but anyone who knows a thing or two about meterology will acknowledge the fact that it is impossible to come up with detailed pressure charts two months or so before a specific day like Nick has and expect to be correct. Whilst the general pattern may be right, Nick's forecast will have to go down as being wrong purely because he is being so specific and he has indicated the importance of this accuracy himself. We can already see that the chart Nick came up with for yesterday was technically wrong, although the overall pattern wasn't too far out. A 1005mb low over Spain where a 1030mb high actually was forecast to be can not go down as being right. At the same time, a 1025mb ridge of high pressure over Ireland is not the same as a 1040mb high sitting there.

I don't begrudge anyone trying to come up with new methods of long range forecasting, but I do feel very strongly that it is a case of trying to run before you can even walk. Drawing pressure maps for 2 months out with self proclaimed high confidence is never going to work, whatever the methodlogy. However, going on and drawing a sequence of pressure maps for a specific time period 2 months out is just a waste of time to be frank, let alone going on to predict an event that is actually probably scientifically impossible to actually occur! No disrespect to you Nick, but it will never work. Stick to general patterns and give yourself some leeway and you may be onto something, but don't get into specifics because you will be fighting a losing battle from day 1.

DITTO Paul B & my sentiments exactly- although I probably worded what you wrote in a far more blunt & quick method-

Unfortunatly there will be people that want it to be right for the wrong reasons- Ie just to try & prove someone wrong & I can feel that 'vibe' in here-

The maps & forecast wasnt/isnt going to happen. Full stop. there maybe some 'light' correlation with general pattern, but thats all- Im quite sure why Roger is wasting all the time & effort to prove what what we already know- good luck with that

S

Edited January 29, 2011 by Steve Murr

[Roger J Smith]

Day 5 validation

____________________

The 5d GFS for 3 Feb has tightened its gradient across the middle of the grid from west to east, and dropped any separate low pressure west of Ireland, moving that trough as a weak feature within the fast WSW flow. Despite some change in appearance from yesterday, the map correlates at .92 with the previous day. There is a slight increase in correlation with MB to 0.19 due to a similar position for the low in Sweden, albeit much weaker on GFS. This is the highest correlation so far between these two. Meanwhile, the trend of moderate correlation GFS with RJS continues on a slow downward trend now sitting at 0.4.

The 5d ECM for 3 Feb has a less intense WSW gradient but is otherwise very similar to the GFS and now correlates at 0.96 with it. We expect to see a convergence on 1.0 in this measure. There were enough changes day 6 to day 5 to keep that statistic at 0.88, and these changes seem to have been away from the MB scenario because the ECM-MB correlation dropped to 0.12 (from 0.32) indicating probably that the Baltic sector has decoupled in the comparison. Meanwhile the ECM and RJS continue about the same moderate correspondence at 0.37 correlation.

The trend then is little change all round except that there are no movements toward either of the experimental forecast outcomes; these do remain somewhat better than random if the model consensus at present time is in fact a good prediction of the eventual reality. The UKMO as well as the GEM five day are somewhat less intense than the ECM and noticeably less intense than GFS in a similar pattern solution. They don't appear to promise vastly different correlation with the experimental forecasts when compared with the ECM.

There is probably about enough time left now for either experimental model to gain or lose about 0.2 in correlation, as I don't imagine that the two main models with this much other support and internal consistency can deviate very far now from their projected solution. But any details on the changes as we head in to time zero will be reflected in where the experimental models finish in the hunt, and then the question becomes, at what time range were they overtaken by the model pack as it closes in on reality (this is pretty well always the outcome for such long-range efforts, a good one might survive the day 7 pack, a poor one might be behind at day 10 -- occasionally you find a case where one matches performance at day 4).

This sort of validation would provide the most useful statistic possible, a "day equal" reading that would give the user an idea of what time range the conventional models would normally produce a map of similar accuracy. Some kind of experimental forecast that achieved a "day equal 5.5" rating would imply equal chances of verification with the major models at 132h. This would be a pretty decent standard and in my own case not one that I would currently claim, my studies to date have indicated results in the range of 7-9 days.

[SMU]

Roger do you not think it would be better to forecast an anomaly pattern as opposed to what the UK will 'generally' see in winter-

For example- forecast a westerly or NW & your results will be high-

Try & use the method to forecast a blocking high across a gridpoint & any decent forecast %age will go down the pan

S

[Pie Man]

if you dont agree or like the thread.why not just go away? i could phrase it more direct but the swear filter may explode.

This is my last post. I thought this forum engaged in genuine weather discussions, not pseudoscience. This forum isn't for me.

Read the pseudoscience link on Wikipedia and take a moment to reflect.

I'm off to the pub to engage in some intelligent conversation.

Goodbye

Pieman

[Roger J Smith]

As a scientific proposition, I don't agree that maps for specific dates could never verify. Clearly this depends on the cause and effect built into any theory of map creation. Let's take 60 days as our standard since 30 days might in some minds represent some potential end-point of the current technology (I feel it's really more like 12 days).

How might one construct a 60-day map and what accuracy could it be expected to display?

One group of theoretical approaches would be statistical. From trends and known starting point pattern driving signals, one could try to crunch the numbers in a form such as a Hovmuller diagram extended out into the 60-day future time range. Accuracy of such a method might intuitively rest at about 3-5 days of error with fairly large problems in pattern recognition. However, what if there were finer signals than we currently recognize, as I think the MJO school of thought tends to believe they are slowly refining. What if you had signals that you could nail down to within 5 deg of lat-long in 30 days, therefore 10 deg within 60 days? Then you might start to have better results in your 60-day maps and achieve a good correlation within 2 days or even 24 hours. I don't think any such system has reached that level of accuracy yet. However, since this is the largest group of researchers with the most research funding and super-computer access, I would not be so rash as to discount the possibility. This would be like getting the CFS model to work at day 3-4 type GFS/ECM accuracy.

Another approach, including my own, is dynamic from external energy sources. This has the promise of revolutionary accuracy since the input is partly astronomical events, and partly solar system magnetic field sectors which have analogue data sets and could in the future have precise space measurements in real time. The problem for this type of approach is theoretical rather than mathematical. If the theory were perfectly developed, the maps generated would be incredibly accurate since there are no statistical degradations of signal inherent in this type of approach. This is a direct sort of equation-driven A implies B kind of map drawing that is oblivious to signals of the conventional sort, although the research when properly done needs to account for those signals. In my own case, I have found that the theory development is a formidable challenge and that even after a long period of time, there is probably a large amount of theory yet to uncover, but signals can be assessed from historical data and I am convinced that this approach is the right way forward and that it has as a theoretical end-point very accurate daily maps at any time scale that the overall grid structure (essentially the geomagnetic field and current large-scale ocean currents) remains intact.

A third approach is pattern matching from past cases. This could be a blindfold version of either of the above, in other words, an unsophisticated researcher who happened to stumble on a working pattern match could be unfolding maps from some past situation because of large-count similarity of factors involved from either menu above. I am not really aware of any successful case of pattern matching beyond seasonal, when people try to do this on a daily scale they usually run into the problems that whichever of the above approaches is chosen, you can't match enough of the variables to sustain a run of daily maps being very similar. In my own research model development I have already eliminated even the possibility of pattern matching on the daily scale within the time range of 300 years, and there are some weak candidates in the 500-800 year time frame but these are useless to us now since we don't have daily data from those distant times, plus the magnetic field has substantially changed since then.

However, pattern matching might work anyway, which would force me to reconsider the variables in my own research model. Even if it worked, it might work at a time ratio that was not 1:1, in other words, if you stumbled across a working correspondence (and this might involve map grid shifting) the following days might diverge slowly due to a slight time differential in pattern evolution. Say this were 1.1 to 1, then day 10 from successful pattern match might involve day 11 from historical analogue set.

Some researchers are trying to solve this problem using only variables in the lunar orbit. I think that will run into major correlation problems no matter how far into the analysis this gets, because the lunar input into the overall shaping of pressure patterns is something like one-quarter to one-third of total external input. Therefore correlations will be held down to a range like -0.3 to +0.3 in such forecasting. There are too many other variables in play for this to work.

I don't think there's any limiting reason to rule out the eventual development of accurate long-term forecast maps. It all depends on what theory advances, and what the time variability functions are with those factors. There's no way to prove or disprove this, but one day we could have the answer. Until that day arrives, correlation studies of current attempts are the only way forward, I don't think any proposed map can be ruled out from first principles, and if the original MB verbal forecasts were in error relative to the maps, that would be a different kind of problem that has nothing to do with map verification. Just look at the maps without reference to numbers for wind speeds or pressures and they don't look so unrealistic (in the abstract, not saying they look realistic as forecasts for dates at this point). I wouldn't want to trash the method because the forecaster was not fully conversant with map interpretation, that's a skill that is more easily learned than 30-60 day forecasting.

And whatever the outcome this is by no means a waste of time for two reasons, first, it gives us a really detailed look at how the models converge on a solution, and secondly I have my own forecast map in the mix and I want to see how it fares against the two conventional models when we look back from 3 Feb to the ten day period of map evolution. So for those reasons alone there is some value in this for me, and if we get a positive outcome for the MB maps then he's got something to build on.

[pottyprof]

My first (and probably my only) post on this thread.

You've made more than one post on here Steve. Can't you remember calling this idea "Jackanory" along with being rather rude about it and also claiming it was the first and only post you'd be making in this thread??

[SMU]

There seems to be the word 'Probably'in there somewhere ....

So the post was a bit blunt--- the thread has meandered along but the bottom lines hasnt changed-

S

[AtlanticFlamethrower]

The models have now moved away from MB's scenario for 3rd February. It's looking windy in the north but very zonal and Westerly. The charts are fairly unremarkable looking and would be a forecast failure were it to stay like this. Perhaps the GFS will trend back to the more MB-like solutions as we get closer to the date.

Edited January 29, 2011 by AtlanticFlamethrower

[Roger J Smith]

People seem to be staking out more extreme positions than I would advise here, basically here's what various validation results would mean (to me) ...

high correlation (over 0.3) would mean keep working on it, you may be on to something

low quasi-random correlation (-0.2 to 0.29) would mean you need to consider much different forcing, this test case has shown no significance

high neg correlation would mean consider your methodology, there may be a systematic error.

Beyond that, it would not mean a lot given one trial (granted MB has produced a series of maps, but the logic remains the same). The detailed correlation study might be a bit of extra work as I'm sure a brief visual comparison would give a pretty good idea of what range the predictions fall into. But this way, you have extra material available, for example, you can correlate parts of the grid to find out where your theory holds up better or worse than your average.

Anyway, I would advise people to just let this happen, and there is certainly no law against trying to make these advances, MB is quite free to keep trying and we can at least assess all further efforts subjectively, if a group of members determines that they think the method has promise, they can form an interest group and go from there. I would look at any further maps but if let's say this correlation and one more prove to be random then I would figure it should await an occasion when a map is obviously non-random to take it up further. So I think that's pretty generous and I don't recall there was this much interest or actual detailed validation when I first brought forward a trial map five years ago. That map actually correlated better than either of these seem likely to do.

As to the chances of getting a good correlation on a blocking pattern, that's probably a sound point and if anyone wants to bring one forward, I will take my hat off to said person -- I've identified some blocking windows of opportunity in the past, but I find it hard to nail down the start and end points, the mid-point might enjoy better success. For example, I think there will be blocking in later February and early March, so if I was challenged to draw up a map, I might go with about 20 Feb to improve my odds. But I wouldn't claim it was a sure thing.

When this period is fully validated and discussed, I would suggest a call for forecast maps for 20-21 March which looks interesting from my perspective. While it's really good to have a long series of maps from MB, we can really get more out of a comparison of several entries if we specify perhaps one forecast time. People can say what they want about uncertainty with their entry. This might give various people some idea how their theoretical approach compares to others, and to state-of-the-art global forecasting in real time.

Had a quick look at 18z, there's nothing there to suggest intensification of earlier trends, perhaps a slight weakening.

[TEITS]

Have to be honest im surprised the discussion is continuing in this thread because its pretty obvious the forecast is going to be wrong.

http://www.wetterzentrale.de/pics/brack4.gif

I can't see no 115mph NW,ly winds over the coast of Holland or an inferior storm to 1953. Certainly don't have to worry about loss of life on the latest fax chart.

Don't get me wrong im not mocking MB forecast but what he has attempted is just impossible. Im all for new and different methods when making long range forecasts but nobody will ever be able to pin point such detail to a particular date. When attempting a long range forecast all you should attempt is to predict whether temps, rainfall, sunshine will be below, near, above average and what the general pressure patterns will be. Basically like how GP presents his forecasts. You will note they aren't vague like the Met O forecasts used to be but on the otherhand he doesn't go into too mucn detail.

The type of forecast that MB has attempted will never be possible even in 2/300 years. Doesn't matter how powerful supercomputers become because there is one factor that prevents humans/computers from making accurate, detailed long range forecasts and that is the chaos theory. Why do you think the models have ensembles? answer is to take the chaos theory into account.

My advice to MB is to continue with the forecasting. However read GPs winter forecast and note how this was presented and use this as a guide in how to present future forecasts.

[Alan Robinson]

A couple of points:

- MB gave a very precise forecast and indicated a very high degree of confidence. Had it been a more general pattern forecast, I doubt there would have been much criticism. Live by the sword, die by the sword or something along those lines...

- No explanation other than something straight out of a New Age pamphlet about the method. Contrary to what some seem to assume, I'm pretty open minded about new ideas but I do resent being thrown a few vapid explanations that seem to be completely at odds with everything we know about our atmosphere and indeed life in general. I did not, nor did any of the sceptics come up with those statements that seem to be completely glossed over in favour of an "everything is possible" approach. I've highlighted them in a previous post.

No one does forecast as precise and as far in time as MB has done using conventionnal methods, demanding of the sceptics to put up or shut up in some kind of competition is neither here nor there. It would be a bit like asking people doubting someone saying he can fly to jump themselves out of the building and do better...

I've had an interest in many things people, some that would even have some of MB's biggest supporters wince in disbelief, and one recurring problem that crops up when you investigate an anomalistic subject is that of "knowledge by arcane", someone claims to holds an essential truth that would give an explanation for the anomaly studied and much more beside but refuses to divulge anything but tantalising and/or bewildering snippets. That leaves the investigator in a quandary as nothing can be tested rigorously. The investigation, even if some or the entirirty of the anomaly is hard to explain in more conventionnal terms, generally dies as we are then in a situation which you could call as "revelation truth" and we enter another realm together, that of belief and faith statements.

Very well put. This coincides entirely with my experiences with gardeners / Rudolf Steiner / Demeter. Arcane is a very fine expression. I think I'll start using instead of esoteric.

[MurcieBoy]

Morning folks, as the ECMWF models are slow coming out, the comparison on this run will be between the Storm Forecast v GFS 30 Jan 0z:

1 Feb

2 Feb

3 Feb

4 Feb

5 Feb

From the 25 Dec 2010 Forecast Appraisal Video (from 4:10 minutes), where I compared the Forecast v GFS, I found the GFS did not "lock in" to the final chart until 2 days before the event. Perhaps the current GFS model has a wee bit more to go before it locks into the 3 Feb 2011 "target day"?

[TEITS]

Sorry MB but after years of following the models i've rarely seen massive changes at the +96 timeframe.

The ECM/GEFS mean plus operational output all suggest that on the 3rd Feb a deep LP system will be centred near Iceland and the UK will be experience W,ly winds which could be strong further N. However what none of them show is +100mph NW,lys down the N Sea and we're unlikely to see the model output change towards showing this especially as 3rd Feb is only +96hrs away.

http://91.121.84.31/modeles/gens/run/gens-21-1-96.png?0

[mountain shadow]

Yup, not going to happen now I'm afraid.

31 pages doon the drain.

[AtlanticFlamethrower]

The low East of Iceland needs to trend further East by a couple of hundred miles and a little South for the forecast to come right. At the moment on GFS, it gets that far then buggers of North.

This doesn't require a major upgrade, but it definitely requires a small trend to move the low Eastwards whereas yesterday the trend was to move the low Westward, lowering the correlation with MB's target day February 3rd.

[Weather-history]

The low East of Iceland needs to trend further East by a couple of hundred miles and a little South for the forecast to come right. At the moment on GFS, it gets that far then buggers of North.

This doesn't require a major upgrade, but it definitely requires a small trend to move the low Eastwards whereas yesterday the trend was to move the low Westward, lowering the correlation with MB's target day February 3rd.

It would need more than this to me, it would also require more amplification of the pattern to get the kind of flow that is being predicted for the UK but it looks very flat. It would need something like early January 1976 with high pressure ridging further north and a distinct gap between the lows but none of the models I have seen are showing this.

http://www.wetterzentrale.de/pics/archive/ra/1976/Rrea00119760103.gif

Infact the pattern reminds me of what happened post Christmas 1994 and New Year 1995 that has been predicted.

http://www.wetterzentrale.de/pics/archive/ra/1995/Rrea00119950101.gif

There is nothing like this in the charts

Edited January 30, 2011 by Mr_Data

[Coast]

Haven't got time to compare, rotate or superimpose these two I'm afraid, I'll try and do something with tomorrows comparisons:

Yup, not going to happen now I'm afraid.

31 pages doon the drain.

Again, as said elsewhere on here, the forecast period isn't over (it has barely begun) and it would be both prudent and courteous to wait until after the 5th February to make such a judgement.

[weatherguy]

To my (I'll admit it) "untrained eye" those charts posted by Coast above do actually look remarkably similar, just that the pattern is a little too far north-west...

Interesting to see if the rest of the forecast comes close in terms of the actual synoptic set up

[Coast]

Sorry, also forgot to include the actual wind speeds, this is for an hour later today, so excuse the hour difference:

[AtlanticFlamethrower]

Slightly better for MB? Red is where it needs to be.

Nothing to get excited about but the way the GFS modelled the 12z PV is quite different to the way it modelled it on the 06z (wasn't there a deep low following this low on the 06z), so there may be slow, subtle but significant changes in timing and position to come.

Not good enough but still enough to keep following the models.

Edited January 30, 2011 by AtlanticFlamethrower