12z Model Comparisons Results

[Thundery wintry showers]

15 May

Weak 1012mb low over southern Britain, and rather cloudy conditions for many with some sharp showers in the south.

GFS scores 6/10; T+144 was poor with a strong ridge from the Azores High giving an anticyclonic/north-westerly flow, T+120 was very accurate, but then T+96 had the low over southern Britain a good 200 miles too far west giving light southerly winds.

UKMO had northerlies at T+144 with a low in the North Sea. It still had northerlies at T+120 with a low to the East, then at T+96 it had moderate westerlies over Britain and the low over the south underdone and a good few hundred miles too far west. Only 3/10; a very poor output.

ECMWF scores 7/10; it was reasonable at T+144 though with low pressure centred slightly too far east, T+120 was accurate, and T+96 had the low slightly too far north, giving westerlies in the south. Overall not too bad.

NOGAPS unavailable at T+144; it scores 7/10 as T+120 and T+96 were both reasonable, T+120 having the low rather too far north giving westerlies in the south.

JMA scores only 2/10; T+144 had a ridge of high pressure over Britain extending from the Azores, T+120 and T+96 both had high pressure to the west, throwing up a ridge over Ireland, with northerlies further east; very poor.

GEM unavailable except at T+120 when it had a ridge of HP over Britain.

Overall a rather poor set of outputs, I particularly can't fathom what's going wrong with the UKMO model at the moment, but that's a number of poor outputs in a row. ECMWF and GFS didn't do too badly though.

[johnholmes]

my current check on whose 'top' from the admirable posts from Ian(TWS) is this, from about 08/05/07.

ECM=4

GFS=3

Met=1

ECM has 1 even with GFS, GFS has 2 evens, the other with the Met O.

it has to be said that the average 'top' mark is not all that good, not sure of an exact figure but below 7 I would estimate.

[Thundery wintry showers]

16 May

1008mb low off NW Scotland, anchored to 996mb low in the Atlantic, westerlies over Britain, 1028mb high west of Spain.

GFS scores only 5/10; T+120 was relatively close with a low north of Scotland and WNW winds, and warm SW'lys coming into the west, but T+96 and T+144 both had the Azores High too dominant, with a strong ridge into southern Britain, pressure some 10-15mb too high.

UKMO scores 6/10; T+144 had a dominant ridge of HP, T+120 was similar to GFS with a low off N Scotland and WNW winds, but T+96 was then reasonably accurate.

ECMWF scores 6/10; at all three timeframes, the HP was too dominant, though less so than with GFS. T+120 was its closest output.

NOGAPS scores only 5/10; Azores High way too dominant at all three timeframes, northerlies for East Anglia, westerlies elsewhere.

JMA scores only 5/10; it was even further out initially, having a prominent Euro High at T+144 and T+120, and SW winds. T+96 was fairly accurate though.

GEM unavailable except at T+96 when it had a low over East Anglia giving N winds for all.

A disappointing set of outputs.

[johnholmes]

Met O now has two E scores one with GFS and one with ECMWF. It has yet to record an outright 'win', ECMWF has 4 and GFS 3 since 08/05/07.

And as Ian keeps commenting none have yet shown a really good score for some time now. I have no idea what is the problem for the models, and I'm sure the 'boffins' at each centre are working busily away trying to resolve the issue.

Edited May 17, 2007 by johnholmes

[Thundery wintry showers]

17 May

972mb low SW of Iceland with extension towards NW Scotland, and SW winds coming into the west of Britain. A ridge of high pressure off eastern Britain.

GFS scores only 6/10; T+144 was okay though the ridge of HP was too dominant, high to the SW too far north, then at T+120 it was quite poor with the low SW of Iceland situated north of Scotland giving westerlies. T+96 was okay again, though underdid the HP ridge to our east.

UKMO had Britain in a chilly northerly flow at T+144. T+120 and T+96 had the Azores High too far north giving Britain an anticyclonic/north-westerly flow. Only 5/10.

ECMWF also scores just 5/10; it had northerlies at T+144, at T+120 it was okay with the Atlantic low too far east giving a general WSW flow for all, and at T+96 it had a Scandinavian Low and Azores High close by giving NW winds.

NOGAPS scores only 4/10; at T+144 and T+120 it was consistent but had Britain dominated by a large high centred just to the south-west, T+96 was unavailable.

JMA scores 4/10 also; slack pressure at T+144, okay at T+120 though with westerlies and LP to the north overdone, then northerlies were shown again at T+96.

GEM scores 6/10; T+120 had the high pressure way too dominant, but then T+96 was probably the most accurate of any output I've seen today.

Another disappointing set of runs. I can't fathom what's going on, but none of the models are convincing at the moment.

[johnholmes]

that makes it 3 each to ECMWF and GFS, but the winning marks Ian is giving is the lowest in a long time. Like you Ian I cannot understand what it is that is making all the 3 main models have such difficulty just now, well since 08/05/07.

[Thundery wintry showers]

18 May

976mb low to the SSE of Iceland sweeping a weak front eastwards followed by sun and blustery showers.

At long last, a decent run from GFS, scoring 7/10. At T+144 the Azores High was too far north with a ridge covering Britain, and low S of Iceland underdone, but T+120 was extremely accurate. T+96 was also good, though the low was slightly overdeepened.

UKMO had the low underdone at T+144, with a warm sector over Britain and SW winds. The low was still underdone at T+120, giving WSW winds, but high pressure clinging onto the southeast. T+96 was reasonably accurate though with the low too far east this time; overall 6/10.

ECMWF scores 8/10; the low was elongated W-E at T+144 giving a long draw of westerlies, but then T+120 and T+96 were very accurate, and at no point did it overdo high pressure to the south.

NOGAPS unavailable at T+120. It was poor at T+144 with low underdone and high pressure over the south, and T+96 had the low still underdone, but also too far southeast with strong cyclonic/WSW winds over Britain. Only a 5/10.

JMA scores 6/10; high too far NW at T+144 giving a warm sector and SW winds; good at T+120, but at T+96 the low was a good 400 miles too far east, though still giving westerlies for Britain.

GEM also scores 6/10; low too far west at T+120 and T+96, giving SW winds for eastern Britain, and the passage of the front delayed.

Overall, a decent set of outputs, with the ECM doing particularly well. UKMO seems to be having its traditional Achilles heel of overdoing Euro Highs and positioning the jet too far north, manifested to a larger extent than usual.

[Thundery wintry showers]

19 May

980mb low north of Scotland giving a strong westerly flow over Britain. Most places had sunshine and showers but it stayed dry over much of East Anglia, the SE and around the Tyne & Wear region.

GFS did very well this time, scoring 8/10. At T+144 the low north of Scotland was correctly placed but a secondary Atlantic low was too far east giving a long draw of westerlies, though with showery regime over Britain. T+120 and T+96 were very accurate with a low N of Scotland giving a cyclonic/westerly flow.

UKMO had low too far north at T+144 and high pressure covering the south, with WNW winds and showers probably restricted to the north. T+120 was better, though still with the low too far north, WNW winds, and relatively high pressure over the south. T+96 was then very accurate; overall 7/10.

ECMWF was the most accurate, scoring 9/10; some minor quibbles about the positioning of the low north of Scotland, but at each timeframe it was very accurate with a cyclonic/westerly flow over Britain.

NOGAPS unavailable except at T+120 when it was fairly accurate, though the low's orientation was wrong.

JMA scores 7/10; accurate at T+144 though with too strong a westerly flow, then T+120 was quite poor with the low to the NE and north-westerlies, then T+96 was reasonably accurate again.

GEM also scores 7/10; low too far north at T+120 with relatively high pressure over the south but otherwise OK, then accurate at T+96.

A much better set of outputs, with the ECM being particularly impressive. UKMO is still lagging behind GFS.

[johnholmes]

'scores on the doors' at the moment from 08/05/07 are

outright wins

ECMWF=6

GFS=4

Met O=0

so ECMWF in the lead over the last 12 days.

[Thundery wintry showers]

20 May

992mb low well to the north of Scotland, slack westerly winds.

GFS was dodgy at T+144, with the Azores High too strong and too far north, giving a strong anticyclonic/WNW flow. T+120 was very accurate, but T+96, though fairly good, had low pressure out in the Atlantic overdeepened giving a slightly stronger W flow than actually happened. Overall this just about makes a 7/10.

UKMO was poor at T+144 with high pressure too dominant over the south. T+120 and T+96 were good, though still with high pressure too dominant over the south, and the trailing front over the far SE (picked up by GFS at both timeframes) was missing. However I give this 7/10 as well; T+96 was closer to being correct than the GFS offering.

ECMWF also scores 7/10; T+144 had high pressure too dominant over the SW with WNW winds, T+120 had too pronounced a showery W flow, then T+96 was reasonably accurate.

NOGAPS scores 6/10; T+144 and T+120 had low pressure too dominant giving a cyclonic showery W flow, and T+96 was unavailable.

JMA scores only 5/10. T+144 had high pressure over Britain, centred in the south; T+120 wasn't much better with a high pressure area off SW England, then T+96 was reasonably accurate.

GEM scores 7/10; reasonable at T+120 and T+96 but with too strong an area of high pressure over Europe.

Some decent outputs in there; JMA was the only model to get it completely wrong at any of the three timeframes.

[Thundery wintry showers]

21 May

980mb low west of Iceland, 1032mb Azores High, slack region of pressure over Britain with light westerlies in the north of Scotland.

GFS scores only 4/10; the low pressure was too far east at all three timeframes (by a good 500 miles or so at T+120), giving strong WSW winds for the whole of Britain.

UKMO scores 5/10; same failings as GFS at T+144 and T+120 but at T+96 the low was comparitively accurately placed, giving strong WSW winds for Scotland and very little wind elsewhere.

Again the ECM outperformed GFS and UKMO, scoring 7/10. T+144 was dodgy, with the low pressure vastly underdone, but T+120 and T+96 were accurate with the low only marginally too far east, giving WSW winds for Scotland and very little wind elsewhere.

NOGAPS unavailable except at T+96 when it was poor, with the low way too far to the SE, giving strong WSW winds for all.

JMA scores 7/10; T+144 was poor with a high to the east of Britain and slack southerlies, but T+120 and T+96 were good, with WSW winds for Scotland and light winds elsewhere.

GEM scores 6/10; T+120 had WSW winds for all and high pressure over the SE, but T+96 was reasonably accurate.

ECM and, surprisingly, JMA were pretty good; GFS and UKMO had problems with positioning low pressure too far east.

[Thundery wintry showers]

22 May

988mb low off northern Iceland, slack westerly flow over Britain, 1032mb Azores High.

GFS scores only 6/10. T+144 was reasonable, though with low pressure overdone to the NW, giving a strong WSW wind for all; T+120 was poor with a low over Scandinavia giving moderate WNW winds, but T+96 was reasonable again, with higher pressure and light winds for the southern half of Britain.

UKMO also scores 6/10; low off N Scotland at T+144 giving strong westerlies for all, T+120 and T+96 were better but they overdid low pressure to the north, giving strong westerlies for Scotland.

ECMWF scores 7/10; low pressure too strong to the north at T+144 and T+120 giving strong WSW winds in the north and lighter winds only in the far south, but T+96 was pretty good.

NOGAPS scores 6/10; T+144 unavailable, T+120 and T+96 okay but had low pressure off N Scotland giving strong westerlies, and high pressure too far south especially at T+120.

JMA scores another 6/10; very similar to UKMO.

Also 6/10 for GEM; similar to NOGAPS.

A very similar set of runs from all the models, with high pressure too far south, low pressure too strong to the north and strong westerlies. ECMWF got closest, but wasn't great either.

[johnholmes]

current marks are

ECMWF correct on 8 occasions

GFS on 4

with the rest shared, 8 between them and Met O, so ECMWF the clear leader sincoe 08/05/07

[The PIT]

current marks are

ECMWF correct on 8 occasions

GFS on 4

with the rest shared, 8 between them and Met O, so ECMWF the clear leader sincoe 08/05/07

Looks like they'll need to do another gfs upgrade.

[Thundery wintry showers]

23 May

1024mb high ridging into south-west England. 996mb low centred over and to the east of Iceland, very floppy looking system. Moderate westerlies affecting Scotland and light winds further south.

GFS was okay, scoring 7/10. T+144 had high pressure too dominant with low pressure centred too far west in the Atlantic, T+120 good but with low pressure slightly overdone giving moderate WSW winds for most, T+96 very accurate.

UKMO overdid low pressure to the north at T+144, giving moderate westerlies for all. T+120 still had too strong a westerly flow with a strong low to the north, but T+96 was reasonably accurate. Also 7/10.

ECMWF scores only 6/10; similar to UKMO but the low pressure overdone to our north to a greater extent at T+144 and T+120.

NOGAPS scores 6/10 also, for at T+144 and T+120 low pressure was overdone to our north giving WSW winds for all; T+96 unavailable.

JMA scores 6/10; fairly accurate at T+144 and T+120 though with disturbances in the flow that didn't materialise, and then T+96 was poor with high pressure in charge for all.

GEM scores 7/10; WNW winds and low to the NE at T+120 but very accurate at T+96.

Again a similar set of runs; ECM in particular had trouble with overdeepening of low pressure to our north, out of the three main models.

I should be able to bring out an updated Excel file by the beginning of June.

[johnholmes]

well that is a change for ECMWF not to figure in the scoring and a change for the Met O to do fairly well.

ECMWF is the clear leader still.

[Thundery wintry showers]

24 May

Low at 996mb over east Iceland, a moderate westerly flow over Britain, high pressure over the south.

GFS scores 7/10 but it wasn't very convincing; T+144 had a low off N Scotland giving westerlies for all and high pressure too far south, T+120 was okay but had Britain between two high pressure zones, giving westerlies over the north and slack winds over the south, T+96 was reasonably accurate but had more of a WSW wind.

UKMO was poor at T+144; low off N Scotland, cyclonic/WNW flow. T+120 and T+96 both had high pressure centred too far north with westerlies only over Scotland and an anticyclone in the south; also 7/10.

ECMWF also scores 7/10; similar to UKMO.

NOGAPS scores only 4/10 based on T+144 and T+120. T+144 had a cyclonic/westerly flow with a low off N Scotland, T+120 had a north-westerly, while T+96 was unavailable.

JMA scores only 6/10; T+144 had low over N Scotland, giving a cyclonic/westerly flow, T+120 and T+96 were okay but had high too far north, more so than GFS, ECM and UKMO.

GEM scores 6/10; it was reasonably good at T+120 though with southern Britain in between highs giving almost no wind, then T+96 had high pressure too far north.

No clear winner this time; the main three models gave very similar outputs.

[Thundery wintry showers]

25 May

Low at 996mb off N Scotland, a westerly flow for all, cooler air spreading south behind a cold front with sunshine and showers for Scotland. The slack 1008mb low off East Anglia was unable to generate the thundery showers that GFS had been projecting.

GFS was decidedly poor; T+144 had a deep low over SW England, giving SSW winds for most, and light winds for northern Scotland. T+120 had Britain in a westerly flow but low pressure was centred over Iceland- a long way too far west. T+96 wasn't any better, with a slack area of low pressure covering the British Isles. Only a 4/10.

UKMO had a low over SW England at T+144 and a strong Icelandic Low with southerly winds and thunderstorms. T+120 and T+96 were better (and also better than GFS) but the low pressure to the N and NE was underdone, and the low off SE England was too far west, covering most of southern England and most likely triggering storms over East Anglia. UKMO scores 5/10.

ECM did slightly better scoring 6/10; T+144 had an Icelandic Low with a slack low over southern Britain, giving a slack airflow, T+120 and T+96 had low pressure quite accurately placed off N and NE Scotland, but overdid low pressure to the south, giving easterly winds for the southern half of England.

NOGAPS unavailable except at T+96 when it was poor, showing southerlies over Britain.

JMA scores only 4/10. T+144 was its best output, with an Icelandic Low, an overdone low over the SE and westerly winds in the north. T+120 and T+96 underdid low pressure to the north and had a thundery low over south-east England giving north-easterly winds for the north.

GEM only available at T+120; shame, because it was actually very good at T+120.

A disappointing performance from the models, the ECM just about secured the highest mark but even the ECM was far from convincing. The low pressure to the south was consistently overdone.

[johnholmes]

ECMWF continues to out perform both GFS and UK Met. UK Met is decidedly poor at the moment.

[NL]

Just a theory but could the May 1st adjustments http://www.weather.gov/os/notification/tin...S_upgrade-2.txt have anything to do with the poor GFS model recently, before May the models tended to be in similar agreement..

admittedly this is from the states but the euro models are out scoring the GFS there too..

Edited May 26, 2007 by Northern Light

[Thundery wintry showers]

26 May

992mb low to the NE of Scotland, a very slack westerly flow, 1008mb low off SW England ready to sweep wet & windy weather in for the remainder of the Bank Holiday weekend.

GFS scores 6/10; T+144 had an elongated deep low off N Scotland giving strong westerlies, T+120 had a deep low off W Ireland (probably an overdeepened version of the 1008mb low to the SW) and T+96 didn't appear to have the low to the SW at all, giving an anticyclonic/north-westerly flow for Britain, but other features of the chart were accurate.

UKMO had low pressure over S Ireland at T+144 giving ESE winds in the north, switching to southerly in the south. T+120 was poor with low pressure off SE England giving NE winds, and T+96 was similar to the GFS at T+96. Only a 4/10.

ECMWF kept vastly overdoing a weak low south of Iceland, and the low off SW England was missing. T+144 had low pressure over the SE also, with easterlies for Scotland, T+120 had northerlies over Britain, and T+96 was fairly okay but with a low over the SE that didn't materialise; overall 5/10.

NOGAPS was unavailable at T+144, had a low over Britain at T+120, and had lows to the NE and NW at T+96 with westerly winds, overall 5/10.

JMA scores only 3/10, having low pressure over the west at T+144 giving southerlies, over East Anglia at T+120 giving northerlies, and northerlies again at T+96.

GEM scores 4/10; T+120 and T+96 both had Britain under a ridge of HP from the Azores with low pressure off East Anglia and another low coming into western Scotland.

Another dodgy set of outputs, with some charts not resembling the actual outcome. GFS was slightly better than the others, but that isn't saying much.

[Thundery wintry showers]

27 May

992mb low over southern England, a strong ENE flow further north. Southern and central areas had a lot of rain, while further north it was brighter with some showers, the latter mainly away from the east coast.

GFS scores 7/10; T+144 had the low too far north, centred over E Anglia, T+120 had it over S Ireland, and T+96 had it accurately placed, the charts weren't too bad overall but the differences in low positioning would have been crucial for the assessment of the Bank Holiday weekend.

UKMO was reasonable at T+144; low only slightly too far west, situated over central southern England. T+120 had the low much too far west (over Ireland), however, and T+96 had an elongated low complex to the south. Also 6/10.

ECMWF scores 8/10; it was better than GFS and UKMO with the low generally too far west, but not by a great deal (centred over Cent S/SW England) and it had the weather pretty much spot on.

NOGAPS unavailable at T+96. On the basis of T+144 (low off W Scotland giving a cyclonic/southerly flow) and T+120 (low fairly accurately placed but massively underdone) I only give it 5/10.

JMA scores 7/10; low placed over central Britain at T+144, but pretty accurate at T+120 and T+96.

GEM scores 6/10; low placed over northern England at T+120, but fairly accurate at T+96.

The ECM was the most consistent in its handling of Sunday although GFS and JMA weren't too bad.

[Thundery wintry showers]

28 May

1000mb low out in the North Sea and NNE winds for the British Isles. There was sunshine and showers in the west, but it was dull and wet in many parts of eastern England.

GFS scores another 7/10; T+144 and T+96 were very accurate, but T+120 was quite a bit out, with the low off NE Scotland giving a generally showery NNW flow.

UKMO had low pressure too far west at T+144, centred over eastern England. T+120 was similar to the GFS at T+120 (low too far north giving NNW winds) but T+96 was good; overall 6/10.

ECMWF also scores 6/10; T+144 was reasonably accurate but T+120 had the low too far west (centred over East Anglia) and T+96 had the low too far north giving NNW winds.

NOGAPS unavailable at T+120. T+144 was dodgy with the low too far north giving NNW winds, but T+96 was very accurate so I give this 7/10.

JMA scores 7/10 also; T+144 and T+96 were accurate though T+96 had the low slightly overdeepened. T+120 was dodgy though with a straight, long-draw northerly flow.

GEM scores only 5/10; low vastly overdone and over East Anglia at T+120 and T+96.

Again a so-so set of scores.

[Thundery wintry showers]

29 May

1000mb lows to the east and west, a north-westerly flow over Britain, with some rain in the north and east.

GFS scores a 6/10; low pressure was much overdone over N Scotland at T+144 and T+96 giving strong westerlies for the north, while T+120 had low pressure off NE England. The general pattern was good, but the charts over Britain decidedly iffy.

UKMO scores 8/10; very accurate T+144 and T+96, slightly dodgy T+120 with low pressure too far south-west, centred just off NE England.

ECMWF scores 7/10; T+144 was poor with a large low over Britain, but T+120 was okay with low pressure slightly too far west giving NW winds, and T+96 was very accurate. Not as good as the UKMO though.

NOGAPS unavailable at T+144; good enough for 8/10 also on the basis of T+120 and T+96, when it was very accurate.

JMA scores 6/10; high pressure too strong to the NW at T+144 with northerlies, accurate at T+120, but T+96 had a GFS-style deep low over N Scotland, giving strong westerlies.

GEM scores 6/10; low vastly overdeepened to east at T+120, and too far west at T+96.

Overall UKMO and, surprisingly, NOGAPS, were very good. A decent set of outputs on the whole.

30 May

1000mb low over N Scotland (rather simiilar to what the GFS showed for 29 May) with 996mb low coming into Ireland.

GFS scores a 7/10; surprisingly this time it rather underdid the N Scotland low at all three timeframes, giving southerlies for all, and low pressure correctly shown as coming into Ireland and spreading rain northwards.

UKMO somewhat underdid low pressure at T+144 giving slack south-westerlies. T+120 had low pressure off N Scotland too far north, and T+96 had an elongated low to the west giving S/SE winds, only 6/10.

ECMWF scores 7/10; T+144 had a vastly overdeepened low off N Scotland but 1000mb low off W Ireland was very accurately placed, T+120 was accurate, but T+96 underdid the N Scotland low.

NOGAPS was unavailable.

JMA had low pressure right over northern Britain at T+144, then at T+120 and T+96 it underdid the N Scotland low, giving southerlies for all; overall a 6/10.

GEM only scores 5/10; weak high pressure at T+120, and underdid low pressure at T+96 giving southerlies.

Overall GFS and ECMWF were the best outputs.

[Thundery wintry showers]

31 May

992mb low south of Iceland, with a SSE flow over Britain. Many areas had sunshine and thundery showers, though easternmost and northernmost parts generally remained dry.

GFS did extremely well, scoring 9/10; it was almost spot on at all three timeframes, just slight quibbles with positioning of the 992mb low (this is the first time I've been able to say this of the GFS for a while!)

UKMO was okay at T+144 but underdid the low off Iceland. T+120 was less good, with westerlies shown over Britain and low pressure far too east, and T+96 had the Icelandic Low much underdone although it had Britain in southerlies. Only 6/10.

ECMWF also scores 6/10; T+144 was good, but T+120 had the low too far east giving a very strong southerly/cyclonic regime, and T+96 had low pressure much too far east, giving westerlies.

NOGAPS underdid low pressure to the west at T+144 and T+120 but was quite accurate with southerlies shown. T+96 was poor, though, with a deep low right on top of Britain, so only 5/10.

JMA kept having Britain in southerlies but positioning the low too far east giving more of a cyclonic regime; a 7/10 overall.

GEM scores only 5/10 as it had a low right on top of Britain at T+120 and T+96; too far east.

An excellent GFS, which much outperformed the other models.