How Did The Models Perform?

[Coast]

With the potential for a bit of interesting weather coming up, I wondered if it would be an idea for those that are interested, to save a few current charts and models on their hard drives and then maybe come back with some analysis of them matched against the real weather in the period being scrutinised in say 3, 5, 7 or 10 days time?

I'm hoping it might provide us learners with a few guides to what was shown and how it turned out, so we can both understand more about models and which ones perform the best in certain scenarios and at anything from a few days to FI.

[Zenarcher]

Going to compare the changes that can occur within 120 hours in different models,

BOM - Has pushed the Atlantic more West and the Russian high has seen a huge upgrade and moved West.

5 days ago

Today

GEM - Didn't do to bad getting things pretty close but on it as well the Russian high has been upgraded.

5 days ago

Today

NOGAPS - Didn't do well at all trying to flatten out things and completely underestimated things to our East.

5 days ago

Today

UKMO - Didn't do to bad but still underestimated the Russian high although not as much as others. Still had a little to much energy over the Northern part of the blocking.

5 days ago

Today

ECM - Looks like it performed the best out of all of them at 120 hours a slight shift to the West and upgraded the Russian high slightly overall good prediction from the ECM.

5 days ago

Today

GFS - Completely underestimated the Russian high by 10mb and the Atlantic lows were being overdone. Everything now has been upgraded to our East and pushed West meanwhile the Atlantic has been downgraded.

5 days ago

Today

[Cloud 10]

A similar example here to the one above,although using northern hemisphere charts from 6 days

ago.

Todays 12z ECM chart is used for verification purposes.

Todays actual chart> GFS 6 days ago>

ECM 6 days ago> GEM 6 days ago>

UKMO 6 days ago>

Seems that the gfs did best on this occasion,as the other 3 models all underestimated

the strength of the blocking high,especially up near Svallbard,although it does have that

random shortwave just west of the UK!

The UKMO is by far the worst performer in this case as it just wanted to smash the block in half.

[Coast]

Thanks for that guys. I am hoping we can build up a profile of the various models performance in certain weather conditions to see how they faired and as a pointer in future discussion when other, similar events crop up

[Neilsouth]

The UKMO is by far the worst performer in this case as it just wanted to smash the block in half.

wow the ukmo did dreadfully, thankyou for posting these charts!. I can't believe how our own uk models could be so low in the performance scale

[Coast]

The UKMO has backtracked these last 48 hours.

UKMO today -

UKMO 2 days ago.

Yes there is a shift west over the UK. But there are bigger changes elsewhere

>Mediterranean low is 10mb deeper, and much further north

>Azores high in particular much further west.

>Heights stronger to our north/north east.

The outlook would be completely different and far colder were this to shift again in a similar fashion.

The GEM looks more plausible to me, and is pretty much exactly what the met office are predicting (albeit slightly further east as they expect the PPN band to allingn across the centre of the country)

Added for further info

[Cloud 10]

Quite a day weather wise for the UK today,so interesting to see what the models were projecting

6 days ago compared to what actually happened.

Again,the ECM 12z chart for today is used for verification purposes.

Todays chart

The ECM,GFS,GEM and UKMO from 6 days ago.

ECM> GFS>

UKMO> GEM>

Good effort by the UKMO,GFS and ECM,which all came pretty close to the actual chart,although i

would give the gold medal to the GFS as it has the trough near the UK a bit sharper

than the other two.

The GEM didn't do so well on this occasion (unfortunately!) as it had the cold block to far west.

[J10]

Of course as well NW keeps charts for 10 days as standard.

So if wanted to look at the profile 10 days ago

to today it is easily possible.

http://cdn.nwstatic.co.uk/gfsimages/gfs.20120205/12/00/h500slp.png (actual today)

http://cdn.nwstatic.co.uk/gfsimages/gfs.20120126/12/240/h500slp.png (10 days ago)

[JonMillar]

Here are the 12z charts from Friday 27th January through to Tuesday 31st January for 12.00 yesterday together with the 12z t+00 for yesterday 5th February

http://cdn.nwstatic..../h850t850eu.png

http://cdn.nwstatic..../h850t850eu.png

http://cdn.nwstatic..../h850t850eu.png

http://cdn.nwstatic..../h850t850eu.png

http://cdn.nwstatic..../h850t850eu.png

http://cdn.nwstatic..../h850t850eu.png

The much maligned Sunday run did push the milder air through too much. It seems it was over correcting from the runs of the previous days. However, the Monday run itself over-corrected in the other direction until at about t+120 it got closer.

I think it just shows the difficulties the models have with dealing with anything over t+120 - t+144 at the moment, and that things will keep swinging around quite violently until you get into the reliable time frame.

Edited February 6, 2012 by JonMillar

[Cloud 10]

Well,according to what some of the models were showing 6 days ago,much of the UK should be in the midst

of a screaming northerly about now,which needless to say hasn't quite turned out as expected. http://cdn.nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/laugh.png

Today's actual chart>

GFS,GEM,UKMO and ECM from 6 days ago.

GFS> GEM>

UKMO> ECM>

Very good performances from the ECM and GEM which both came very close to the actual chart,a fairly woeful effort from GFS and UKMO both of which projected a fairly potent northerly.The UKMO continued with the northerly for another day or so before finally coming in to linewith the other models.Interesting to note that the FAX charts also showed a northerly for a couple of days,even when nearly all guidance went against it.

Edited February 14, 2012 by Cloud 10

[johnholmes]

Very good performances from the ECM and GEM which both came very close to the actual chart,a fairly woeful effort from GFS and UKMO both of which projected a fairly potent northerly.The UKMO continued with the northerly for another day or so before finally coming in to linewith the other models.Interesting to note that the FAX charts also showed a northerly for a couple of days,even when nearly all guidance went against it.

an interesting set of charts.

I am all in favour of this kind of checking it does help to get a 'feel' for the models and with different set ups. If we based our views just on the NOAA 5 and 6 day checks then this kind of detail does not show up. Also our memories tend to be rather selective.

keep it up please.

[Cloud 10]

An interesting day of weather across the UK today with an active cold front sweeping south followed in the north

by some squally rain/sleet/snow/hail showers with the odd rumble of thunder thrown in,and turning progressively

colder through the day,so what were the models showing 6 days ago and how have they verified?

Today's actual chart>

ECM,GFS,GEM and UKMO from 6 days ago.

ECM> GEM>

GFS> UKMO>

Overall a very good performance from the models on this occasion with all four very similar to the actual

chart,the GEM not quite as accurate as the other three.

[Cloud 10]

Some very warm temperatures across parts of England and Wales today,with possibly some new records

being set,so was this set-up predicted well by the models?

Today's actual chart.

The ECM,GEM,GFS and UKMO from 6 days ago.

ECM> GEM>

GFS> UKMO>

Another very good performance by all the model's,with all of them having high pressure just to

our south-west feeding in some very mild air.

Hard to pick a winner,maybe the UKMO.

[Zenarcher]

Going to see how the models predicted today at 144 hours ago.

Today's actual charts

144 hours ago (23rd Feb)

GFS - Didn't do to well underestimated pressure to our East and failed to pick up the low in the Atlantic.

ECM - Done much better with high pressure to our East still underdone it slightly also the low to the West of the UK never came to but it did pick up the low in the Atlantic and done well on its positioning.

UKMO - Had things looking rather flat and underdone the high to our East like the GFS. It did like the ECM pick up on the low in the Atlantic.

JMA - Done very well with handling pressure to our East and it did pick up on the low as well if it had the low a touch more North it would be pretty much bang on the mark 144 hours out.

NOGAPS - Done pretty well with things to our East and did in a way pick up the low in the Atlantic.

GEM - It done the best overall well done GEM it managed to get the hang of everything correctly with high pressure to our East and the positioning, deepness of the low in the Atlantic.

[Cloud 10]

Plenty of weather action across the UK today with just about everything bar the kitchen sink,so what

were the models predicting for today?

Today's actual chart.

GFS,GEM,UKMO and ECM from six day's ago.

GFS> GEM>

ECM> UKMO>

A fairly good performance from the models with each of them having the trough over the UK

and the high pressure to the north-east.

ECM to strong with the low pressure over the UK,the UKMO maybe the closest.

[Cloud 10]

The words "high" "pressure" and "dominant" seem to be a recurring theme (some might say a stuck record ) in

this week's model output discussion,so how well did the model's perform?

Today's actual chart.

The ECM,GEM,GFS and UKMO from 6 days ago.

ECM> GEM>

GFS> UKMO>

A very good performance from all four model's with high pressure dominating!!!

Edited March 11, 2012 by Cloud 10

[Zenarcher]

Going to see how the models done at 120 hours with today's high pressure over the UK and the Atlantic low.

Actual Pressure Chart For Today

GEM - Done well with the low in the Atlantic but underestimated the high pressure.

JMA - Very similar to GEM.

NOGAPS - Done very well with the Atlantic low but completely under done the high pressure.

UKMO - Done very well on both the low and high pressure.

ECM - Very similar to the UKMO near identical.

GFS - Our winner it handled the low and the high pressure the best nearly perfect at 120 hours.

1. GFS.

2. UKMO.

3. ECM.

4. GEM.

5. JMA.

6. NOGAPS.

(UKMO and ECM pretty much a draw).

[Cloud 10]

Heavy snow and strong winds outside as i type this,so how well did the model's do in predicting this rather

abrupt return to winter.

Today's actual chart.

The ECM,GEM,GFS and UKMO from six days ago.

ECM> GEM>

GFS> UKMO>

The ecm and ukmo did well on this occasion with both of them getting the N/NE airflow correct,although

neither picked up on the strength of the shortwave/low pressure over the country at the moment but

still a very good effort for 6 days out.

A fairly poor effort by the GEM,and especially the GFS which seemed to miss the boat completely

this time!

[Cloud 10]

GFS not doing well at the moment,must try harder!

[Cloud 10]

Some decent action for convective fan's across part's of the UK today,so how well

did the models do in predicting these April showers?

Today's actual chart.

And the ECM,GEM,GFS and UKMO from 6 days ago.

ECM> GEM>

GFS> UKMO>

All models correctly predicted low pressure affecting the UK,although the clear winner here is the GFS which

came very close indeed to the actual chart.

The other three models had the low pressure over the UK as more of a cut-off feature.

[johnholmes]

GFS not doing well at the moment,must try harder!

yes that has to be the worst validation I've seen in a long time. Neither ECMWF nor Met are free from 'wobbles' though as the graph shows a short while ago.

[Cloud 10]

yes that has to be the worst validation I've seen in a long time. Neither ECMWF nor Met are free from 'wobbles' though as the graph shows a short while ago.

The gfs has "pulled its socks up" over the last week or so.

Interesting to note how poor the models have verified for the southern hemisphere

on the graph,not sure why this is?

[Cloud 10]

Some very unpleasant weather across many parts of the UK today with heavy rain and strong winds

and some pretty cold temperatures all brought about by low pressure moving up from the south,so

how well was this predicted?

Today's actual chart.

The ECM,GEM,GFS and UKMO from six days ago.

ECM> GEM>

GFS> UKMO>

Quite a difficult set-up for the models to get right with a low moving up from the south,but a good performance from

the ECM,GEM and GFS.

The UKMO not bad,although it did overdo the high pressure to the north of us.

[loafer]

Not strictly a model (although models involved), it is worth revisiting the Met's April-June forecast...

"The forecast for average UK rainfall slightly favours drier than average conditions for AprilMayJune as a whole, and also slightly favours April being the driest of the 3 months."

http://www.metoffice.gov.uk/media/pdf/p/i/A3-layout-precip-AMJ.pdf

[Cloud 10]

Quite a decent day across many parts of the UK today thanks to a ridge of high pressure coming in

from the west,although some of us still managed to experience showers!

Today's actual chart.

The ECM,GEM,GFS and UKMO from six days ago.

ECM> GEM>

GFS> UKMO>

A varied performance from the models with the GFS seeming to do the best,whereas the ECM had a rare off day

with it showing a small low over the south which would have delivered a rather different day if it had verified.