Model Moaning Thread...

[Shiny_Bottom_1]

Thought it might be worth opening a new thread on this.

It seems everywhere I turn in the winter forum, its 'Unreliability' this, and 'FI' that.

But in all seriousness, why do we blame so much on the models?

The models are, purely (from what I understand anyway), an interpretation of data that is fed into them. The initial data is real time, >T+0 is how the computer models the data (based then on other factors, I guess, such as historic data).

However, we seem to think that the computer can interpret the data into a run that is flawless. Well, I think its time we realise that they are not flawless, but in addition they never have been.

I guess the frustration comes from the computers modelling some good, dare I say cold and/or snowy runs, and then downgrading nearer the time? Well, I would say, more than guess, and be probably 90% sure that is the case.

Isn't about time we accept the models for what they are, and perhaps be thankful we have access to such a wide variety of data? Or am I barking up the wrong tree here, and only except data which has a >95% accuracy rate?

[johnholmes]

I suspect, sadly, Chris that you may have opened a can of worms.

Like you say we should be happy to have so much access for free, equally as you say we should accept the models for what they are, and leave their improvement to those better able than any of us to make improvements. This would be so much preferable to the various doom and gloom posts we get on the model thread. Quite why it descends at times into personalities I am at a loss to explain.

We, fortunately here, live in a pretty free and easy going society(not perfect by any means but not bad compared to some)so each of us is entitled to say what we prefer most and its human nature to try and 'bend' what is shown on the models to what we may prefer. Why then start to name call and worse.

If we did not it would certainly make moderating this site far easier, and no I'm not a mod.

[Shiny_Bottom_1]

I suspect, sadly, Chris that you may have opened a can of worms.

Hopefully John Would be good to perhaps get this cleared up a little into some of the comments that float about.

and no I'm not a mod.

A rocker perhaps

[SMU]

What annoys me is how the models become totally flawwed when it comes to projecting cold spells & yet become amazingly accurate when it comes to predicting a warm spell-

It would seem that there must be some historic emphasis or bias into projection conditions especially the further out you go as cold spells always seem longer than they actually are-

The ECMWF has a better grip on ptterns & theres a whole peice on the website--- Stick with that- the GFS is becoming cannon fodder against it-

S

[The PIT]

What annoys me is how the models become totally flawwed when it comes to projecting cold spells & yet become amazingly accurate when it comes to predicting a warm spell-

It would seem that there must be some historic emphasis or bias into projection conditions especially the further out you go as cold spells always seem longer than they actually are-

The ECMWF has a better grip on ptterns & theres a whole peice on the website--- Stick with that- the GFS is becoming cannon fodder against it-

S

I presume you're referring to Models and not what users say and want to see. Dunno I'll think you find that in Summer there's just a much swapping and changing over hot spells and thundery weather. It's just that you won't get many complaints over a Hot Sunny day at 25C when the Models originally predicted 33C. People still can enjoy the weather. We also don't probably look at how mild spells evolve and change because in Winter we're not really looking for them just the cold spells. So the Cold spell gets all the interest while the mild run gets another rubbish run comment. The GFS 12oz isn't far behind the ECM which is the best performer. I still wouldn't call the GFS cannon fodder.

[Rollo]

Seems to me that mild weather being much more zonal is easier for the models to predict whereas picking up an easterly (certainly in winter) is not hard to see coming but an awful lot can and does go wrong before it comes to fruition.

Perhaps the historic information just is not there in enough quantity re the easterly but is there in huge quantities when it comes to zonality.

[chionomaniac]

What annoys me is how the models become totally flawwed when it comes to projecting cold spells & yet become amazingly accurate when it comes to predicting a warm spell-

It would seem that there must be some historic emphasis or bias into projection conditions especially the further out you go as cold spells always seem longer than they actually are-

The ECMWF has a better grip on ptterns & theres a whole peice on the website--- Stick with that- the GFS is becoming cannon fodder against it-

S

I couldn't agree more. It has got to the point that whenever the GFS projects any cold of any substance it will look good, be upgraded and have everyone rubbing their hands with glee upto 4-5 days away. Then differences between the GFS and other models start to occur until 2-3 days out. We all think how can the GFS backtrack now having shown this projected cold spell for so long, but backtrack it does until T+0 when the -5 isotherm brushes south for 5 minutes.

I think I would now be more surprised if a cold spell predicted at around T+96 came off than if a back track occurred!

Perhaps one day a cold snap will be shown with the -5ºc isotherm at T+120 for 5 minutes and this be slowly upgraded so that by T+0 the whole country is under -10ºc air with no end to it in sight....Dreaming again.

c

[SP1986]

What annoys me is how the models become totally flawwed when it comes to projecting cold spells & yet become amazingly accurate when it comes to predicting a warm spell-

It would seem that there must be some historic emphasis or bias into projection conditions especially the further out you go as cold spells always seem longer than they actually are-

The ECMWF has a better grip on ptterns & theres a whole peice on the website--- Stick with that- the GFS is becoming cannon fodder against it-

S

Not sure I entirely agree re: ECM to be honest, I think each model is good for a particular setup, in the setup thats imminent, as we saw 2 weeks ago, the ECM didn't handle it too well, whereas the GFS seemed to handle it well, therefore this probably why milder spells tend to be recieved better in FI (because more people are watching the GFS)

[PB22]

One other thing to bear in mind is that people don't tend to carefully study the models apart from when a cold snap/spell is progged, so many assumptions and conclusions about the models are drawn from how they perform when predicting a cold spell.

[Earthtrasher]

I think it needs to be more widely appreciated that model projections are not predictions.

[karyo]

One other thing to bear in mind is that people don't tend to carefully study the models apart from when a cold snap/spell is progged, so many assumptions and conclusions about the models are drawn from how they perform when predicting a cold spell.

You have just nailed it on the head! Obviously as most people are looking for snow/cold they (or we to be fair) tend to focus on that. When the cold is downgraded we always notice. By contrast mild spells don't get as much attention and whether we get 12c or 9c at T0 nobody notices.

Karyo

[Shiny_Bottom_1]

Hi Karyo. The other side of what Paul mentions, re: people not carefully studying the models, is that most take the control or op run as gospil. They dont look for intra-run* agreement, or inter-model* agreement.

John H recommends a very good model watch, but focusing on one run.

*I think i've made these terms up, but I hope you get the drift.

[Thundery wintry showers]

My view on the main models (ECMWF, GFS, UKMO) is that they're usually accurate out to T+96, but accuracy declines at T+120 and T+144, and once you get past T+168 they are good only for trend-spotting.

There tends to be more errors when slight differences in low/high positioning make a large difference to the weather over Britain, which is why the models are often considered to do better when the weather is anticyclonic- if a large high ends up 200 miles further east than suggested 6 days beforehand, it tends not to make much difference. Whereas sometimes, it can be the difference between a northerly and a southerly.

In my 12Z model analysis I found the ECMWF to be slightly more reliable than UKMO and GFS, but not because it consistently got the outcome right, rather because when it went wrong, it had less of a tendency to go wildly wrong than GFS. That's what we saw to an extreme in the latest cold snap- ECMWF slightly overdid the northerly, but nowhere near as much as GFS did. UKMO tended to be about as accurate as ECMWF at T+96 and T+120, but the T+144 outputs tended to be inaccurate, often overdoing high pressure to the east and south.

As for the models' individual strengths, UKMO was generally best at handling situations with a block to the east, GFS best at handling mid-Atlantic blocks and zonal situations (though with a bias towards cold zonality and overdeepening of lows), while ECMWF tended to be quite consistent across the board.

[reef]

The models have the same amount of chopping and changing no matter whether its cold or mild. The difference is that cold in the UK is always marginal (well at least it is these days). This tends to mean that where a 850hPa temperature of -7C might give snow, a temperature of -5C might not. In the grand scheme of things this 2 degree difference is absolutely nothing. Its probably likely that this could be applied to mild spells aswell, but no-one is going to notice the difference between 12C and 14C in winter nor snow at -18C instead of -20C in Canada or Siberia.

We also have to take into account that its much harder to get cold in the UK than mild, everything needs to be just right. So much so that if something isnt right, then you can pretty much write off cold and snow events for 95% of the population. It seems every year theres a new-type of spoiler. We've now got a stronger jet fuelled by higher SSTs which tracks further north on average each decade. That same jet takes longer to subside so any wintry spells come later and later, January is almost a write off now, being particularly prone to this as SSTs are not at their coldest like in February/March. For the same reason you have shortwave development, delaying the time it takes for cold to dig in. Lets not forget the polar vortex and stratospheric temperatures, another side-effect of more energy being in the system.

It is very much a case of perception, though the models are far from perfect. Im suprised however that they do so well in light of how our climate is constantly changing.

[karyo]

Hi Karyo. The other side of what Paul mentions, re: people not carefully studying the models, is that most take the control or op run as gospil. They dont look for intra-run* agreement, or inter-model* agreement.

John H recommends a very good model watch, but focusing on one run.

*I think i've made these terms up, but I hope you get the drift.

Hi Chris,

I agree with what you say. Many people tend to instantly react on what a model run with either excitement or disappointment without further investigation. Focusing on one run (say the 12z output) is a good idea but like an addict, I tend to view every output (0,6,12,18z) in the winter. In the summer, I am more likely to just view one output a day as I don't find the weather as interesting.

Karyo

[davehsug]

I may be completely wrong but haven't I read that GFS is very very good at the Southern hemisphere? If so that would suggest that it's much better at handling developments over water than over land. The fact that it comes out 4 times a day to t384 also seems a dreadful waste of resources. However good the model, even at an experimental level, that sort of range is not within our capabilities at the moment & probably never will be.

Having said that, I detect a definite drop off in performance over our area of the world in the last few months. In it's defence though, it is a GLOBAL forecasting system & we just happen to live in a small corner of the world, which seems to have the most unpredictable weather of all!

Dave

[Thundery wintry showers]

The last bit of info I read suggested that ECMWF has a slight lead for the Northern Hemisphere, with GFS and UKMO a close joint second (which is also the conclusion I arrived at from the 12Z model comparisons).

Meanwhile, for Southern Hemisphere ECMWF was well out in the lead with UKMO second and GFS a poor third.

[Shiny_Bottom_1]

Interesting to see that the people replying to this thread, are those who post constructive posts regarding the model outputs.

Strange that all those that clutter the model discussion thread up, havent posted in here yet?

[davehsug]

Interesting to see that the people replying to this thread, are those who post constructive posts regarding the model outputs.

Strange that all those that clutter the model discussion thread up, havent posted in here yet?

Reverse psychology Chris! perhaps this could become the new "secret" model thread Thanks for cleraring up the Southern hemisphere thing TWS I knew it was one or the other & as usual I plumped for the wrong one

Dave

[J07]

Personally I think the models are wonderful things, and they don't seem to get a lot of respect. I think they do a good job for what they are. But of course they will never be perfect. I'm not sure people realise the extreme non-linearity of the atmosphere! Open any textbook specific to meteorology and what do you see? Non-linear, non-linear, non-linear equation after equation after equation! And even these can just be approximations. There are many "fudges" going on when constructing a mathematical model of the atmosphere, some work pretty well but others can get quite strained. But if you don't use the "fudge" you may end up with something that is entirely unsolvable by analytic means.

That's where, thank God, we have computers. I wish I knew the ins-and-outs of the models and the actual mathematical differences between them, but I don't. However, you can get your bottom dollar that they do a heck of a lot of solving of highly, highly non-linear differential equations, and they do it numerically. These two things combined are never going to give you perfect results! Some simple looking equations are unsolvable analytically- consider the integral of exp(-x^2) dx. Simple looking, but cannot be solved without using erf(x), which I think (correct me if I'm wrong) is only defined numerically, not analytically. To be honest, I dread to think the absolute horrors that those poor computers are solving, second by second, minute by minute.

And the thing is, because they do this, they take heaps of calculation off the hands of people, allowing people to get on with what they are good at - THINKING*!

...hence pushing the science to higher and higher levels, which in turn goes on to improve the models, which in turn goes on to give us better results!

In other words, I think these computer models are great. People are learning all the time- this will improve the computers.

For a good example of non-linearity, the pendulum is easy enough. It's non-linear, but can be "fudged" to become linear. However, as this is an approximation, it's going to go wrong at some point. I don't know the details of that system in particular, but it's certain that at some finite (and possibly very large) time t, the fudged solution will diverge significantly from the actual behaviour. Now consider the atmosphere, how much more complex it is. Computer models aren't going to improve exponentially, they can only do it gradually, bar an absolutely mind-blowingly enormous breakthrough in analytic analysis of non-linear equations (not to mention chaos....partial differential equations....and so on).

* I should also mention moaning.