12z Model Comparisons Results

[J10]

17th July

ECM (96): High Pressure over UK 1025mb correct. Greenland/Iceland Pressure well dealt with, slightly lower pressure west of UK reasonably dealt, low pressure Scandinavia too far west. 7/10

ECM (120): High Pressure over UK 1025mb correct but poorly defined. High Pressure Greenland correct, and low pressure off Norway was forecasted over Iceland. Timing is out on this forecast 5.5/10

ECM (144): High Pressure over UK 1025mb correct and excellently positioned. Slightly low pressure to west of UK spotted. High Pressure Greenland underdone, and low pressure off Norway was too deep and to south. Good Forecast for T+144 8/10

7/10 Overall.

UKMO (96): High Pressure over UK 1025mb correct but poorly aligned. Greenland/Iceland Pressure well dealt with, slightly lower pressure west of UK missed, low pressure Scandinavia too far west. 6/10

UKMO (120): High Pressure over UK 1030mb slightly overdone but poorly defined. High Pressure Greenland missed, and low pressure off Norway was forecasted over Iceland. Not that good 5/10

UKMO (144): High Pressure over UK 1020mb too low and reasonably positioned, High Pressure Greenland underdone, and low pressure off Norway was too deep and to south. 6/10

6/10 Overall

NOGAPS (144): High Pressure over UK 1030mb slightly overdone but poorly defined. High Pressure Greenland missed, and Gin reasonably forecasted 7/10

No Overall Score

GFS (96): High Pressure over UK slightly overdone and poorly aligned. Greenland/Iceland Pressure well dealt with, slightly lower pressure west of UK too far East, low pressure Scandinavia too far NW. 5.5/10

GFS (120): High Pressure over UK 1025mb correct but poorly defined. High Pressure Greenland correct, and low pressure off Norway reasonably forecasted. 7/10

GFS (144): High Pressure over UK 1025mb correct and reasonably positioned. High Pressure Greenland well forecast, and low pressure off Norway was too deep and to south. 7/10

6.5/10 Overall

GEM (96): Pressure over UK main body 1025mb correct but alignment all wrong, slightly lower pressure west of UK spotted. Low pressure over Iceland instead of West of Norway, Not good for T+96. 4/10

GEM (120): Pressure over UK main body overdone and alignment all wrong, slightly lower pressure west of UK spotted. Low pressure over Iceland instead of West of Norway and Greenland High missed. 5/10

GEM (144): Pressure over UK correct and alignment reasonable, slightly lower pressure west of UK spotted. High pressure correct over Greenland and low pressure west of Norway too far west. Overall a good effort for T+144. 8/10

Overall 6/10

JMA (96): Pressure over UK main body 1025mb correct and reasonably aligned, slightly lower pressure west of UK missed. Low pressure over Iceland in error, Low West of Norway too low, but Greenland High well done. 5/10

JMA (120): High Pressure over UK 1030mb slightly overdone but poorly defined. High Pressure Greenland correct, and low pressure off Norway was forecasted over Iceland and far too developed. Not that good 5/10

JMA (144): High Pressure over UK 1025mb too far East. Low Pressure around Iceland in error effects High pressure far too much. High Pressure over Greenland well predicted. 5/10

Average 5/10

[J10]

18th July

Low Marks for this one as it was a very difficult and messy picture to predict. None of the T+144 were even close to the final outcome, but at least the ECM and GFS had a very good stab at T+96, but the GEM completely got the chart wrong on all 3 attempts.

ECM (96): High Pressure over Greenland correct, Low Pressure SW of Iceland, slightly over developed. Low Pressure SW of UK not developed enough, and High Pressure over UK slightly too strong but good alignment. But a very good effort. 8/10.

ECM (120): High Pressure over Greenland reasonable, Low Pressure SW of Iceland, not recorded. Low Pressure SW of UK not developed, and High Pressure pattern over UK reasonable. 6/10.

ECM (144): None of the main features are in the right place, but there is High Pressure around UK. 4/10.

6/10 Overall.

UKMO (96): High Pressure over Greenland correct, Low Pressure SW of Iceland and SW approaches slightly over developed. and High Pressure over UK slightly too strong but good alignment. But a very good effort. 8/10.

UKMO (120): None of the main features are in the right place, but there is High Pressure around UK. 4/10.

UKMO (144): None of the main features are in the right place, but there is High Pressure around UK and a ridge around Greenland.4/10.

5/10 Overall

NOGAPS (96): The General pattern of the High Pressure is right, but the alignment is still well out. Low Pressure SW of Iceland is positioned too far East and the Greenland High is missing. A generous 5/10.

NOGAPS (120): None of the main features are in the right place, and High pressure is totally wrongly aligned. 3/10.

NOGAPS (144): None of the main features are in the right place, and High pressure is totally wrongly aligned. 3/10.

4/10 Overall

GFS (96): High Pressure over Greenland correct, Low Pressure SW of Iceland, slightly over developed. Low Pressure SW of UK too far East, and High Pressure over UK slightly too weak. But a very good effort. 8/10.

GFS (120): The High pressure over UK is there but not aligned correctly high pressure also correct for Greenland Low Pressure SW of Iceland is not there, but slight low pressure West of UK is sort of there. 5/10

GFS (144): None of the main features are in the right place, but there is High Pressure around UK and a ridge around Greenland.4/10.

6/10 Overall

GEM (96): None of the main features are in the right place, and High pressure is totally wrongly aligned. A very poor 3/10.

GEM (120): None of the main features are in the right place, and High pressure is totally wrongly aligned. Very similar to the T+96 3/10.

GEM (144): None of the main features are in the right place, and High pressure is totally wrongly aligned. Very similar to the T+96/T+120 3/10.

Overall 3/10

JMA (96): The General pattern and the pressure over the of the High Pressure is right, but the alignment is still out. Low Pressure SW of Iceland is positioned too far East, but the Greenland high is well dealt with. 6.5/10.

JMA (120): None of the main features are in the right place, but there is High Pressure around UK. 4/10.

JMA (144): None of the main features are in the right place, but there is High Pressure around UK and a ridge around Greenland.4/10.

Average 5/10

[Thundery wintry showers]

Thanks for the assistance- much appreciated.

To be honest, I prefer your format to my old one- I think I'll revert to it in future.

[Thundery wintry showers]

Thanks in no small part to help from Julian, here's the 20th July results.

20 July 2006

Small 1010mb lows in the Atlantic, one just off W Scotland, with slack SW flow over Britain, large elongated 1020mb high extending from Scandinavia into southern Europe producing a long drag of southerlies into the near-Continent.

GFS

144: Wrong, high pressure at 1025mb shown over N Scotland and lows shown way to the south.

120: An improvement, with 1015mb low shown over western Britain and slack southerlies for the east, the low a little too far south and underdone, but a decent attempt.

96: Unavailable.

Based on the T+144 and T+120 I award this run 5/10.

UKMO

144: Completely wrong, with high pressure over NW Scotland, and the Atlantic low shown to the SW of Britain.

120: An improvement, with the low shown at 1015mb over western Britain, but still a bit underdone and too far south, and 1030mb high shown to the west of Scandinavia.

96: An improvement again, but with the low still a bit too far southeast, and underdone, giving Britain a slack regime with low pressure at 1015mb.

I rate this run at 5/10.

ECMWF

144: Wrong, 1025mb high shown over N Scotland and lows well to the south.

120: Still poor, slack pressure at 1020mb shown over the south, 1025mb high for the north.

96: Much better, 1010mb low slightly to the SE of where it actually ended up, but otherwise the pattern was correct, with southerlies for eastern Britain.

I award this only 4/10- the T+96 was good, but T+144 and T+120 were both very poor.

NOGAPS

144: Wrong, 1030mb high over N Scotland, lows well to the south.

120: Quite good, the pattern was correct, though the lows to the west were underdone, giving slack southerlies over the whole of Britain.

96: An accurate output, with the low at 1010mb just off W Scotland and southerlies over most of Britain, westerlies in the south.

I give this 7/10.

JMA

144: Same mistake as the other models- high at 1030mb over N Scotland, lows to south, easterly winds.

120: Still poor- easterlies shown, lows closer, covering southern Britain, high pressure to north.

96: Lows still underdone, positioned too far west, with slack 1020mb pressure over Britain, though starting to get the pattern right.

Only 3/10.

GEM

144: Poor, 1030mb high shown off western Britain and anticyclonic northerlies, very little detail also.

120: Better, but 1015mb low still too far south, only over SW England, easterlies elsewhere.

96: More accurate, but low a little too far south, with cyclonic southerlies over Britain. Quite a good attempt overall with the elongated 1020mb high accurately placed.

I award this 6/10.

It looks like the models have been having problems recently from the looks of the 15-18 July summaries- and 20 July wasn't any better. Surprisingly, NOGAPS was the most accurate of the models this time.

[Thundery wintry showers]

21 July 2006

Slack 1020mb high covering southern Britain with a very slack W flow elsewhere. 1005mb low in the Atlantic.

GFS:

144: 1015mb low shown over the British Isles, with high pressure too extensive to the north, though the Atlantic low was correctly placed.

120: Still 1015mb low shown over Britain, about as good/bad as the T+144.

96: Pretty accurate.

I rate this at 6/10.

UKMO:

144: 1015mb low shown over western Britain with southerlies for the east, low in Atlantic not shown.

120: Pretty accurate, 1015mb low to N slightly underdone, but 1020mb high correctly shown in the south, 1010mb low in Atlantic.

96: Accurate again.

This scores 7/10.

ECMWF:

144: Very poor, 1025mb high shown over N Scotland and lows to the S.

120: Better, but Atlantic low too far east, giving a relatively cyclonic southerly flow over Britain.

96: Accurate.

Overall, this run scores a 5/10.

NOGAPS:

144: 1015mb low shown over Britain.

120: Accurate, Atlantic low slightly overdone, but generally slack westerly over Britain with 1020mb high over the south.

96: Accurate.

This run scores 7/10.

JMA:

144: 1025mb high shown over Scotland and easterlies- as poor as the ECMWF.

120: Accurate, Atlantic low slightly overdone but displaced a bit to the west, so the flow over Britain was pretty much spot on.

96: High too dominant, at 1020mb extending across the whole Britsh Isles.

Only 5/10.

GEM:

144: Poor, high over N Scotland and easterlies.

120: Rather odd chart, with 1015mb weak low shown over N and W Britain and a cyclonic SW flow for the SE.

96: Better, but Atlantic low shown as extending to N of Scotland with pronounced W winds.

Only 3/10 for the GEM.

The UKMO and NOGAPS produced the best outputs, with the ECMWF, JMA and more especially GEM having problems.

[Thundery wintry showers]

22 July 2006

1010mb low over W Scotland and a cyclonic SSW flow over Britain, which turned out to be conducive to thunderstorms. Very slack pressure to the east.

GFS:

144: A good attempt, the only significant error being the 1010mb low a bit too far west, giving higher pressure over Britain than actually happened.

120: Poor, with 1020mb high shown over Britain.

96: Poor, with 1020mb high shown over Britain.

Only 4/10, even though the T+144 was good. The later outputs were poor.

UKMO:

144: 1020mb high shown over Britain.

120: 1020mb high shown over Britain.

96: Finally a decent output, though with the Atlantic low still too far west, giving a similar scenario to the GFS at T+144.

A score of 4/10.

ECMWF:

144: Atlantic low with an extension to N Scotland giving pronounced W flow, and 1020mb high over south, not very accurate.

120: 1020mb high shown over Britain.

96: Low still too far west, with southerlies for W Britain, and 1020mb high for SE England.

This run scores 4/10 also.

NOGAPS:

144: Way overdeepened 1000mb low off W Scotland with strong cyclonic SW winds shown.

120: 1020mb high over southern Britain.

No T+96, so this run gets a 4/10.

JMA:

144: 1020mb high over south, westerly winds in north, with overdeepened low to west.

120: 1025mb high over Britain.

96: 1025mb high close by, generally high pressure at 1020-1025mb, low way out to west.

Only 3/10.

GEM:

144: 1015mb low shown over the south, high pressure over N Scotland.

120: 1020mb high over SE England, strong cyclonic SSW flow elsewhere, with low in Atlantic overdeepened at 1000mb.

96: 1020mb high in the south, alignment poor with pronounced SW flow shown originating in the Azores, elongated low to W.

This run scores 4/10 also

A poor set of runs for the 22nd July.

[Thundery wintry showers]

Posting marathon starts here. I couldn't be bothered to write a few at once in order to catch up, leaving a backlog to do today!

A new scoring system is in place, which should refine it a bit, borrowed from Julian's ideas. It will be a weighted mark- T+144 counts 1, T+120 counts 2, and T+96 counts 3 (as it is a worse reflection on a model if it can't get it right nearer the time).

23 July 2006

1010mb low off NW Scotland, slack westerly flow over Britain, 1020mb high into the extreme south in the form of a ridge from the Azores High.

GFS:

144: Poor, with 1020mb high shown over the British Isles and the Atlantic low well out to the west. (3/10)

120: Same as T+144- rather poor. (3/10)

96: This one was much better- had low only slightly underdone, and at 1015mb to the NW and W. It had a slack W'ly over Britain and 1020mb high to the south. (8/10)

Overall 5/10- T+96 was good, but the other two were poor.

UKMO:

144: Low too far west, giving a southerly flow for Britain and 1020mb high to the E as well as S, though it was closer than the GFS at T+144. (4/10)

120: High at 1020mb correctly shown over S Britain, but low pressure still too far west- virtually no wind shown across the north, 1010mb low a good 300 miles too far west. (6/10)

96: 1020mb highs to the north and south, Britain sandwiched in between, 1010mb low even further out to the west. (3/10)

This scores only 4/10, it was never really correct, and got worse as we got nearer the time.

ECMWF:

144: Disappointing, 1020mb highs to both the north and the south, similar to UKMO at T+96. (3/10)

120: 1020mb high shown over Britain. Similar error to GFS. (3/10)

96: 1020mb highs to the N and S, low way out in the Atlantic. (3/10)

Overall 3/10.

NOGAPS:

144: Not too bad- main low too far W, but a 1015mb small low just off W Scotland and slack WSW winds over Britain, 1020mb high over the south.

No other timeframes available, so no mark.

JMA:

144: 1025mb high shown over Britain. Poor. (2/10)

120: Still 1025mb high shown over Britain. (2/10)

96: Decent- low to NW only slightly underdone, at 1015mb, and 1020mb high over the south, with slack westerly winds shown. (7/10)

I still mark this down at 4/10, because the T+144 and T+120 were extremely poor, and T+96 was not without its faults.

GEM:

144: Low overdone and too far NW, with cyclonic SW winds shown, high pressure clinging onto the SE. (4/10)

120: Low correctly placed, high a bit too far E with a displaced 1020mb centre, resulting in more of a SW flow. (7/10)

96: No low to the NW, but 1020mb high correctly placed. Almost no wind across the north. (5/10)

This scores 5/10.

Overall, a strong tendency was for the models to overdo the high pressure, placing it too far northeast, and to underdo the Atlantic lows. All models were poor, as seems to have been the case this July.

[Thundery wintry showers]

24 July 2006

The 24th July had a 1020mb high covering most of Britain, 1010mb low to the west, westerlies for N Scotland and southerlies for Ireland. I remember it being a hot sunny day.

GFS:

144: Low pressure ridiculously overdone at around 990mb to the west, but positioned too far west, such that there was still a 1020mb high over Britain. (6/10)

120: Pretty good, with the high correctly placed, lows also not too bad, though main low a bit to the west of where it ended up (7/10)

96: High slightly underdone, but another reasonably accurate output, 1020mb high over the south, slack pressure over the north. (7/10)

This scores 7/10.

UKMO:

144: High overdone a bit, positioned too far north, but 1020mb high over Britain and low to the west, so the general pattern was correct. (7/10)

120: Low too far SW, but 1020mb high still correctly placed over the British Isles. Not a bad effort. (7/10)

96: High grossly overdone, with 1020mb high covering the whole of north-west Europe and Atlantic lows well out to the west of where they should have been. (4/10)

Only 5/10. The T+144 and T+120 were reasonable, at good 7/10 standard, but the T+96 ruined it.

ECMWF:

144: Similar to GFS, low somewhat overdone and too far W, but 1020mb high over Britain was correct (6/10)

120: 1020mb high correctly over Britain, if somewhat overdone with a large extension to the north (7/10)

96: Unavailable.

The ECMWF scores 7/10 on the basis of the outputs available.

NOGAPS:

Only T+144 available. It was quite accurate, but had the low to the W overdeepened and too far W, much as with GFS and ECMWF.

JMA:

144: Low to W underdone and too far west, 1025mb high over Britain, so high a bit overdone but in the right place (6/10)

120: Reasonable, low to W too far SW and a 1020mb high also shown to the north, but 1020mb high correctly positioned over Britain. (7/10)

96: Inaccurate, with 1020mb high shown to the south, and a low bringing a more unsettled W/NW flow to northern areas (4/10)

Overall 5/10.

GEM:

144: High at 1020mb too far east, with 1015mb low over NW Scotland, and a slack SW'ly (4/10)

120: 1020mb high over Britain, lows in broadly correct places (8/10)

96: Low to W too far SW, highs at 1020mb to the N and S and Britain sandwiched in between, though the general pattern was correct (6/10)

Overall 6/10.

A reasonable performance by both the GFS and the ECM.

[Thundery wintry showers]

25 July 2006

1020mb high over eastern England, 1015mb low to the west, incredibly slack regime with a light southerly flow.

GFS:

144: Notably accurate, with the high and low both correctly placed, and other features on the chart also good (9/10)

120: 1015mb low too far south, with SE winds shown over Britain, high to the NE rather than E (6/10)

96: 1015mb low too far east, suggesting a less settled outlook than actually happened, though still with southerlies over Britain and the general pattern good (6/10)

This run scores a 7/10.

UKMO:

144: Poor, 1025mb high to the north and easterlies, low way away to the SW (3/10)

120: Poor again, still high to the north and easterlies, and low too far south (4/10)

96: Quite accurate, low a bit far east, but southerlies correctly shown over Britain, and slightly better than the similar GFS T+96. (7/10)

Overall 5/10.

ECMWF:

144: High at 1020mb over Britain, rather than to the east, with low to W too far west, though the pattern was recognisable. (5/10)

96: An accurate output, low to W slightly underdone but otherwise spot on (8/10)

There was no T+120, so based on those two, the model's mark is 7/10.

NOGAPS: Available only at T+144, but again, the T+144 output wasn't too bad, with the low to W overdeepened, but otherwise accurate.

JMA:

144: Poor, 1025mb high to north and easterlies (3/10)

120: 1020mb high over Britain, with if anything a slight northerly drift in eastern areas (3/10)

96: Lows too far south, high too far north, giving easterlies for the south, but at least the high was to the east of us, and pattern was close to being correct (7/10)

Overall 5/10.

GEM:

144: High at 1020mb over Britain, everything too far west (4/10)

120: High to north at 1025mb, easterlies, low too far south, positioned to the SW (4/10)

96: High at 1020mb to north, low at 1015mb to SW, and SE winds shown (5/10)

Overall 4/10.

The ECMWF and GFS again did okay, but the other models continue to struggle with this rather slack regime.

[Thundery wintry showers]

26 July 2006

990mb low to the NW, 1020mb Azores ridge to the SW, but very little of note over Britain- neither low nor high pressure, hardly any wind, with pressure in the 1015-1020mb range.

GFS:

144: Low to NW too far south, 1025mb high to north, 1010mb low to south, and ESE winds (2/10)

120: Low pressure overdone with 1010mb low shown over western Britain, low in Atlantic still too far south, but closer to being correct (5/10)

96: Reasonably accurate (8/10)

Overall 6/10.

UKMO:

144: 1025mb high to north and easterly winds (2/10)

120: Quite accurate, 1010mb low shown off NW Scotland, but otherwise good (7/10)

96: Not as good, with 1020mb low shown to the north and light easterlies (5/10)

Overall 5/10.

ECMWF:

No T+144.

120: Pretty good, with slack pressure shown over Britain, low in right place to NW (8/10)

96: Very accurate (9/10)

Overall, since I don't have T+144, I'll give this an 8/10.

NOGAPS:

144: 1025mb high to north, pronounced easterly flow over Britain (2/10)

120: 1020mb high to NE and southerlies (5/10)

No T+96, so NOGAPS scores 4/10 overall.

JMA:

144: 1020mb high over Britain, slight easterly flow for south, low to NW nowhere to be seen (4/10)

120: 1020mb high to NE, SSE winds, low to NW too far south (5/10)

96: Quite good, slack pressure, 1020mb high shown to the north (7/10)

Overall, 6/10.

GEM:

144: 1025mb high to north, easterlies (2/10)

120: 1020mb high to N, low to SW and SE winds shown (4/10)

96: Very accurate (9/10)

Overall this run scores 6/10.

The models seemed to like the scenario of HP to our north, and easterly winds to begin with, but all tended towards the slack-pressure scenario. ECMWF, although only two timeframes available, was the best performer, followed by the GFS.

[parmenides3]

I've just 'found' this thread and it's great! Thanks TWS. Have you been keeping a running score, surely you must have? vis a vis John Holmes' thread, which model is 'winning' at the moment?

[Thundery wintry showers]

27 July 2006

Still a fairly slack picture, 1015mb low over the north and a slack westerly flow, 985mb to the NW of Ireland.

GFS:

144: Too much of an elongated low to the north, with westerly winds (5/10)

120: Reasonably accurate, low to north slightly overdone at 1010mb, light westerly winds shown (7/10)

96: Pretty much as T+120 (7/10)

Overall 7/10.

UKMO:

144: Decent, low to NW slightly too far south, low to N slightly overdone at 1010mb but slack westerly regime correctly shown (7/10)

120: Not as good, 1010mb low shown to south and slack easterly regime, though low to NW correctly placed (4/10)

96: Very accurate (9/10)

Overall 7/10.

ECMWF:

144: 1010mb low shown over the east, with cyclonic slack northerlies (4/10)

120: Very accurate indeed (9/10)

96: Again, very accurate (9/10)

Overall 8/10.

NOGAPS:

144: High to NE overdone, with easterlies shown, though with low to NW correctly placed and other features okay (5/10)

96: Fairly accurate (8/10)

T+120 was unavailable, I rate it at 7/10.

JMA:

144: 1020mb high to north, easterly winds, similar to NOGAPS but with low to NW too far south (4/10)

120: Still Scandinavian High overdone, with easterlies for the north, westerlies for the south (5/10)

96: Unavailable.

This therefore scores 5/10.

GEM:

144: 1005mb low to SW, cyclonic SE winds (2/10)

120: Quite accurate, low to N overdone at 1010mb (7/10)

96: Decent, but low to NW too far east, pumping fronts into Ireland (7/10)

Overall this scores 6/10.

A better performance from the models, with ECMWF scoring highest. The Excel file of the current standings will be updated once I get up to date, which will hopefully be this evening.

[Thundery wintry showers]

30 july 2006

I didn't have access to the chart for 12Z on 28 July 2006, so for now I'll skip to 30 July.

990mb low to the west of Ireland and Scotland, 1005mb extension to the north, with moderate westerlies over Britain, and sunshine and a few showers mainly for the N & W.

GFS:

144: Not good, with lows in completely the wrong places. There was a deep low at 990mb to the SW, and a 1005mb low to the NE. The only good thing I can say is that it had westerlies over Britain (3/10)

120: Okay, though it overdid the low extension to our north (showing it as 995mb) and so the orientation of the westerly flow was wrong, though it did have us in a showery westerly (6/10)

96: Almost faultless (9/10)

Overall 7/10.

UKMO:

144: Poor, 1015mb high over the south, 1000mb isolated low to the N, slack lows to the W, and a slack, anticyclonic type westerly regime. (3/10)

120: Not bad, but had the low too far west, giving us a cyclonic SW'ly (6/10)

96: Underdid the low pressure, giving us a cyclonic SW'ly (7/10)

A mean of 6/10.

ECMWF:

144: Slightly underdid the low to the NW, but otherwise a very accurate output, with us in slack W/WSW winds (8/10)

120: Less good, had 1010mb trough over the S, and the low too far W, giving cyclonic S/SW winds elsewhere (6/10)

96: Decent, but continued to underdo the low pressure, giving us a cyclonic SW flow and higher pressure in the south (7/10)

Overall 7/10.

NOGAPS:

144: Low too far W, with a SW flow over Britain and high pressure clinging onto the SE (5/10)

120: Similar, but with a southerly flow shown over Britain due to incorrect orientation of the low, which was still too far west (4/10)

96: Quite accurate, had the low to our N slightly underdone resulting in a slacker westerly flow (8/10)

Overall, a score of 6/10.

JMA:

144: Low massively underdone, just a very slack westerly, low pressure to the N and NW at 1010mb (4/10)

120: 1015mb high ridging into the SE with SW winds, low too far west (5/10)

96: As T+120 but with the low even further west (4/10)

Overall, only a 4/10.

GEM:

144: Slack low pressure to the N, main low too far W, and slack westerlies (5/10)

120: Low pressure too far south and underdone, with westerlies in the south, slack winds in the north (5/10)

96: Decent, but lacking in detail, low to NW a bit underdone, light to moderate WSW winds (7/10)

Overall, 6/10.

The GFS produced by far the most accurate output at T+96, however ECMWF was the most consistent, so the two both managed scores of 7/10.

model_comparisons.doc

model_comparisons.xls

[Thundery wintry showers]

31 July 2006

995mb low off NW Scotland, cyclonic SW winds, and a mix of sunshine and showers.

GFS:

144: General pattern correct, but had secondary lows incorrectly shown. 5/10.

120: Decent, but 1005mb low to the NW underdone, so a slack showery SW flow shown. 7/10.

96: Not bad, but had everything shifted about 200 miles too far east, with Britain in showery westerlies. 6/10.

Overall 6/10.

UKMO:

144: Almost perfect. 9/10.

120: Everything a bit too far east, lows underdone, so slack showery westerly shown. 6/10.

96: High pressure too close to SE, more of a straight SW flow, low a bit far west. However, general pattern was correct. 7/10.

Overall score: 7/10.

ECMWF:

144: Similar to UKMO's T+96- low a bit far west, and high into SE, but close to being correct. 7/10.

120: As UKMO T+120. Slack showery westerly. 6/10.

96: Reasonably accurate. 8/10.

Another 7/10.

NOGAPS:

144: Low underdone and too far north, cyclonic SW winds and 1020mb high into the south, but close to the correct outcome. 7/10.

120: Low too far SW, with cyclonic SSW flow for northern areas, SW flow for the west and south, but quite close. 7/10.

96: Quite good, too "straight" a SW flow rather than a cyclonic one. 7/10.

Overall 7/10.

JMA:

144: Low to NW correctly placed but underdone, showery SW winds (7/10)

120: Low too far south, with southerlies shown (5/10)

96: Decent, with showery WSW winds shown, a bit too much of a secondary low shown to the SW (7/10)

Overall 6/10.

GEM:

144: General pattern correct, had secondary low over western England though. 7/10.

120: Low too far east, with a showery westerly. 7/10.

96: As T+120. 7/10.

Overall 7/10.

A very similar performance from the models- not great, but not bad either.

1 August 2006

1000mb low off NE Scotland with strong cyclonic NW winds. T+96 unavailable.

GFS:

144: Very accurate, low slightly underdone at 1005mb, otherwise perfect. 8/10.

120: Not very accurate, low too far west, secondary low shown to SW, westerly winds. Only 5/10.

Overall, 6/10.

UKMO:

144: Slack NNW winds, low at 1010mb, way underdone (6/10)

120: Low too far west, showery WSW flow shown (5/10)

Overall 5/10.

ECMWF:

144: Quite good, low too far north, but cyclonic NW regime correct (7/10)

120: Low too far west, over N Scotland, with showery westerly, but quite close to being correct (7/10)

Overall 7/10.

NOGAPS:

144: Low underdone and too far W, with cyclonic showery westerly (6/10)

120: Low too far west, with secondary lows shown (4/10)

Overall 5/10

JMA:

144: Low too far west, showery westerly, but other features generally correct (7/10)

120: Poor, elongated lows to N shown, secondary lows (4/10)

Overall 5/10.

GEM:

144: Decent, low a bit far west, but correctly showing cyclonic W/NW flow, overall about as close as GFS got (8/10)

120: Another decent output- low a bit west but cyclonic NWs correctly shown (7/10)

Overall 7/10

The ECMWF and GEM performed best on this occasion, JMA and NOGAPS had problems with elongation of the low to our north and bringing in secondary lows.

[Thundery wintry showers]

4th August

High at 1025mb off SW England, anticyclonic north-westerly winds, northerly in eastern England. It was generally dry but in many places it was rather cloudy. T+144s unavailable.

GFS:

120: Quite a good output. Pressure was too high to the north, though, so a pronounced N flow in the east, and little wind elsewhere. 7/10.

96: Very good, only error being high a bit too close. 8/10.

Overall, a score of 8/10.

UKMO:

120: Remarkably accurate- nothing to fault over Britain. A low to the East was about 100 miles too far south; that's the only difference I could find. 9/10.

96: Unavailable.

Unfortunately, I can't give the UKMO a mark; it was looking good.

ECMWF:

120: Decent, high a bit too far SW with lower pressure over Britain (7/10)

96: As T+120 (7/10)

Overall, 7/10.

NOGAPS:

120: Similar to GFS, quite good, but pressure too high to the north, allowing slack winds away from the east. 7/10.

96: Entire Grid Undefined.

Sadly, no mark here either.

JMA:

120: Not a good output- high pressure too far to the SW, with pronounced northerlies across the whole of Britain. 5/10.

96: Strange 1015mb low over the SW, with a northerly regime established, 4/10.

Overall only 4/10.

GEM:

120: High too far south, slack westerlies, low pressure to north. 5/10.

96: As T+120. 5/10.

Overall 5/10.

Hopefully tomorrow, with a full set of data, will provide more results!

[Thundery wintry showers]

5th August

All three timeframes are available, with the exception of the WZ "Entire Grid Undefined"s.

1025mb high off SW England and a slack northerly regime for England, Wales and Ireland but with strong south-westerly winds powering into Scotland with a broad upper pool of warm air, 995mb Icelandic Low.

GFS:

144: Had the high a bit too far SW though with the correct orientation, and the low too far W. The slack pattern over England & Wales was correct, but it didn't have the south-westerlies powering into Scotland. Pressure was about 5mb lower for most than actually happened. I give this a 6/10.

120: Still had the main Icelandic Low a bit too far west, the high was correctly positioned but, at 1020mb, slightly underdone. Again the output was correct for England & Wales, but the SW winds affected only N Scotland. 7/10.

96: High slightly to the SW of the actual outcome, otherwise accurate. 8/10.

Overall score 7/10.

UKMO:

144: Although the main low was too far west, it had another depression over Iceland. The high pressure was perfectly positioned, with slack winds for England & Wales, and SWs for Scotland. 9/10.

120: Unavailable.

96: Accurate again. 9/10.

The UKMO scores 9/10.

ECMWF:

144: Too progressive, Icelandic low too far east, and high too far south, with westerlies over Britain. 4/10.

120: Decent, but still overdid the low pressure, high too far south. Slack W winds for the south, SWs for Scotland. 7/10.

96: Pretty accurate, but had northerlies for eastern England because a North Sea low was overdone, but otherwise spot on. 8/10.

Overall 7/10.

NOGAPS:

144: Not too bad, but overdid the low pressure and positioned it too far east, slack WSW winds in south, cyclonic SWs for Scotland. 6/10.

120: Poor, with a low to the east overdone, giving northerly winds for most, Icelandic low too far W and high too far to the SW. 3/10.

96: Unavailable.

Overall 4/10.

JMA:

144: Poor, with a low to E shown, Britain sandwiched between highs to the N and S, and lows out to the west. 4/10.

120: Poor, high too far west, northerlies over most of Britain. Icelandic Low way too far SW 3/10.

96: A good output, though it overdid the low pressure, with slack W winds in the south, cyclonic SWs for Scotland. 7/10.

Overall 5/10.

GEM:

144: High too far south, with a moderate westerly over Britain, but Icelandic Low correctly placed. 6/10.

120: Quite accurate, high still a bit too far south, but had the SWs into Scotland and light winds elsewhere. 8/10.

96: High too far NE, Icelandic low too far SW, so England & Wales was correct but it didn't have the SW winds for Scotland. 6/10.

Overall 7/10.

The UKMO was the most accurate of the models. Unfortunately the T+120 for UKMO was unavailable, because of the "Entire Grid Undefined" problem, but the T+144 and T+96 were both more accurate than any other model produced. GFS, ECMWF and GEM were okay, but NOGAPS and JMA had problems.

[Thundery wintry showers]

6th August

995mb low to the NW of Iceland, 1025mb high to the SW, slack anticyclonic westerly flow over Britain. The weather was dry and sunny over most of England & Wales.

GFS:

144: Not a bad effort, but the high too far west, which threw everything a little. A nose of 1020mb high pressure shown over Britain. 7/10.

120: Quite close again, but the high again too far west, low pressure more dominant this time thanks to a secondary low shown off N Scotland that didn't materialise. 6/10.

96: No T+96.

The overall score is 6/10.

UKMO:

144: No T+144.

120: Similar to the GFS T+120, with high too far west and secondary low erroneously shown off N Scotland, but other features accurate. 6/10.

96: Unavailable.

I can't rate the UKMO for 6 August.

ECMWF:

144: Very good, high only slightly to the south of where it ended up, Icelandic Low too far west, but pattern over Britain spot on. 8/10.

120: Similar to T+144, so another 8/10.

96: Let the model down a bit, with secondary low off N Scotland, not as pronounced as on the UKMO or GFS T+120, but still giving a more unsettled outlook than was the case. 7/10.

Overall 7/10.

NOGAPS:

144: Low too far south, low pressure Scotland, unsettled westerlies for Britain, didn't even resemble the correct outcome. 3/10.

120: Same mistakes as at T+144. 3/10.

96: Entire Grid Undefined.

Overall, 3/10.

JMA:

144: Overdid the high and underdid the low off Iceland, giving a very anticyclonic outlook for Britain, although all features were correctly placed. 6/10.

120: Quite accurate for Britain, orientation of high slightly off but near-perfect positioning of everything. 8/10.

96: As T+120, so 8/10.

Overall 8/10.

GEM:

144: Too cyclonic, high and low too far south, cyclonic WSW winds over Britain. 4/10.

120: High too far east this time, though close to being correct. 7/10.

96: As T+144- too cyclonic, low/high both too far south. 4/10.

Overall, 5/10.

A good run from the JMA, with the ECMWF also rather good.

[parmenides3]

Thank you for posting those attachments. Have you told John Holmes?

P

[Thundery wintry showers]

7 August

1025mb high to the west, a northerly flow for the eastern side of Britain. 995mb low over Svalbard (the islands to the NW of Norway)

GFS:

144: High a bit too far south, and low dominating over Iceland, giving a more pronounced anticyclonic NW flow, though not terrible. 6/10.

120: No T+120

96: A 9/10 for this one, it was excellent.

Overall 8/10.

UKMO:

144: Low pressure to the north appears non-existent, but high pressure correctly placed and airflow over Britain very good, so overall a good output. 8/10.

120: No T+120.

96: High ridging too far north and too far west, with pronounced NNE flow in eastern areas, but okay. 7/10.

Overall 7/10.

ECMWF:

144: Pretty good, high a bit too dominant and too far north, but a good output on the whole, even the low to the north was correctly placed. 8/10.

120: Another reasonable output, similar to T+144. 8/10.

96: Similar to the previous two, so 8/10 again.

Overall 8/10- very consistent, very good.

NOGAPS:

144: High too far west- strong northerly winds over the whole of Britain. 4/10.

120: Decent, high too far east, almost on top of Britain, but other features were good. 7/10.

Overall 6/10, as T+96 was "Entire Grid Undefined".

JMA:

144: High too far southeast, low pressure to the NW, but with the high correctly ridging into southern Britain. Quite a few errors so I say 6/10.

120: A good output, similar to the ECMWF's efforts- 8/10.

96: Another good 8/10

Overall 8/10.

GEM:

144: High too far SW with anticyclonic westerlies over Britain, 5/10.

120: High too far SW, pronounced NW flow for Britain, another 5/10.

96: High too far NE this time, covering most of Britain, NW winds for Scotland. 6/10.

Overall 5/10.

A very good set of outputs today. The ECMWF was the most consistent, the GFS produced the best output of all, but the JMA after a dodgy T+144 was also good.

[Thundery wintry showers]

Thank you for posting those attachments. Have you told John Holmes?

P

John knows about it, though I can't be certain that either of them worked for him (I know the Excel one doesn't; that's why I posted the Word one as well)

8 August

1000mb low to the east of Iceland. 1030mb Azores High to the SW with a ridge into southern England, strong westerlies for most. It was quite a sunny day in the south and east, but rain spread southeast.

GFS:

144 unavailable.

120: Low too far west, and high pressure shown over the whole British Isles, low pressure too far away to have any effect. Not a good output, 5/10.

96: Low still a bit too far west, but getting there, 7/10.

Based on those two outputs, 6/10.

UKMO:

144 unavailable as well.

120: Both low and high too far north, with northerly winds for eastern Britain and high pressure over the W and SW, just 6/10.

96: Worse again, with the high and low too far west as well as north, and northerlies over Britain, 5/10.

Overall 5/10.

ECMWF:

144: UKMO-type errors, high and low both too far north, NE winds shown. 5/10.

120: Low and high again too far north, but with the UK over slack pressure rather than northerlies. 6/10.

96: Everything marginally too far north, but this time very close to being correct, high only over the south, WSW winds for the north. 8/10.

Overall, a score of 7/10.

NOGAPS:

144: High a bit too far north, but also displaced a bit to the east, so westerlies were correctly shown over Scotland, high pressure in the south. I say 7/10 for this one.

120: UKMO/ECMWF style problem, but not as acute, with westerlies into N Scotland, northerlies for eastern Britain, 6/10.

96: Unavailable.

Overall 6/10.

JMA:

144: High way too far north, with NNE winds of origin a long way north in the east, high pressure in the west. 4/10.

120: Decent, GFS-style problem of putting the low and high too far west, slack pressure over Britain, 7/10.

96: Pretty good, 8/10.

Overall 7/10.

GEM:

144: The only model that actually overdid the low pressure at any point. Low too far south, with cyclonic WSW winds for NW Scotland which are shown as very strong, moderate WSWs elsewhere, high a bit too far south, but the general pattern was close. I say 6/10.

120: Another 6/10- low too far east this time, giving westerlies.

96: Pretty accurate, 8/10.

Overall 7/10.

The ECMWF, GEM and JMA performed comparably well, but the GFS and UKMO and NOGAPS had problems with positioning all features a bit too far north and/or west.

[johnholmes]

hi Ian

John knows about it, though I can't be certain that either of them worked for him (I know the Excel one doesn't; that's why I posted the Word one as well)

where is the Word one please? Is it a recent one?

cheers

John

[Thundery wintry showers]

I haven't included attachments recently- the latest one is 30 July 2006, which has a Word attachment.

[johnholmes]

ok cheers

[Thundery wintry showers]

9 August

1000mb low off NE Scotland and strong WNW winds over the British Isles. The weather was quite bog-standard, with cloudy windy conditions for many although some areas had sunny intervals.

GFS:

144: Low too far to the NW, with anticyclonic north-westerly winds over Britain, though the Azores High was in the right place. The only error on the whole chart, albeit a major one, was the displaced low. 5/10.

120: Low slightly too far west and slightly underdone at 1005mb, but a much better attempt- WNW winds over the British Isles. 7/10.

96: Low slightly underdone but very close to being perfect, 8/10.

Overall 7/10.

UKMO:

144: High pressure too dominant with anticyclonic northerlies and the high pressure firmly in charge, 4/10.

120: Worse still, low pressure nowhere in sight, NNE winds on eastern flank of large high pressure. 3/10.

96: Low pressure too far north, finally showing a westerly regime over Britain, but an anticyclonic one. 5/10.

Overall 4/10.

ECMWF:

144: High pressure in charge, with anticyclonic northerlies. 4/10.

120: Similar to UKMO T+96, low too far north and anticyclonic westerlies. 5/10.

96: A very accurate output, 9/10.

Overall 7/10.

NOGAPS:

144: High pressure far too dominant, almost on top of Britain, 3/10.

120: A decent output, low a bit too far north but NW winds shown over Britain, 7/10.

No T+96, so I award NOGAPS with 6/10.

JMA:

144: Decent, but the low a bit too far north, pronounced anticyclonic WNW winds. 6/10.

120: High pressure still too dominant, low too far north, but closer- more of a cyclonic WNW flow shown. 7/10.

96: High pressure too dominant, about the same as T+144. 6/10.

Overall 6/10.

GEM:

144: Low too far east, anticyclonic WNW winds over Britain and high pressure a bit too far NE. 5/10.

120: High rather more dominant, almost touching SW England (4/10)

96: High in right place, low in right place but massively underdone (1015mb), so only 7/10. Moderate NW winds shown.

Overall 6/10.

The common trend among the models was to overdo high pressure and underestimate the intensity of the low that moved south-eastwards to the north of Scotland. The UKMO was most guilty of this, persisting with a high-pressure outlook. The GFS was most consistent at getting the low vaguely positioned correctly, though it was the ECMWF that, after a poor T+144 and T+120, had it nailed on at T+96.

[Thundery wintry showers]

10 August

Not much change from the previous day, 1005mb low off eastern Scotland and a cyclonic, pronounced NW flow over the British Isles.

GFS:

144: Overdid the low's southerly extent, positioning it just off East Anglia with N/NE winds for Britain, the error in positioning of the low threw the whole output. 5/10.

120: Reasonable, the low a little bit too far west resulting in less of a cyclonic NW flow, but nonetheless it correctly had us in NW'lys. 7/10.

96: A 9/10 for this one, excellent.

Overall a score of 8/10.

UKMO:

144: UKMO still persisting with having high pressure out to the west of us and anticyclonic northerlies, no low pressure in sight, 3/10.

120: Even more anticyclonic, though signs of low pressure gathering north of Scotland, still 3/10.

96: Finally picking up the trend, low slightly too far west but a good cyclonic WNW flow shown, 8/10.

Overall 5/10.

ECMWF:

144: Anticyclonic northerlies and high pressure, 3/10.

120: High pressure still too dominant, low pressure identified but further north than on the GFS T+120, with an anticyclonic north-westerly shown. Still it resembled the correct outcome, so 6/10.

96: Very good, 9/10.

This run scores 7/10.

NOGAPS:

144: High pressure too dominant, anticyclonic NW winds, low pressure too far north, 5/10.

120: No T+120 or T+96, so no mark. NOGAPS keeps coming up "Entire Grid Undefined" these days.

JMA:

144: Not too bad, high pressure dominant but the low only a little too far northeast, with cyclonic NW winds for Scotland, pressure too high in the SW, 6/10.

120: Worse, low overdeepened and too far NW, with anticyclonic westerlies, 5/10.

96: Low underdone and too far north, anticyclonic NNW winds, 5/10.

Overall 5/10.

GEM:

144: Only a 3/10 for this one, with the high almost sat over southern England, anticyclonic WNW winds, no sign of low pressure.

120: Low in more or less the right place but wrong orientation giving westerlies over Britain, but still not a bad effort, 7/10.

96: Worse, low too far north and westerlies, high pressure close to the S, 5/10.

The GFS was the most accurate all-round, though the ECMWF was a close second. The UKMO didn't pick up on the correct pattern until T+96, while the JMA and GEM dithered about never really locking onto the correct pattern.