12z Model Comparisons Results

[Thundery wintry showers]

23 June

A slack 1008mb low over northern Britain. Western areas were dry and fairly sunny while eastern areas had heavy prolonged showers and, for many, thunderstorms.

GFS was a dodgy run, scoring just 5/10. T+144 was pretty accurate, though with low pressure too far east giving NNW winds, but then T+120 and T+96 were poor, with low pressure centred off eastern England, giving northerlies for all.

UKMO also scores only 5/10- low pressure was off to the east at all three timeframes, giving a straight northerly flow.

ECMWF scores 6/10- it was better, with more of a north-westerly regime at T+96, though T+120 and T+144 were not so good with high pressure overdone to the SW and low pressure out in the North Sea.

NOGAPS unavailable except T+120 when it was extremely poor (low coming into western Britain bringing SSW winds)

JMA scores 6/10- poor T+144 and T+120 with low pressure out in the North Sea giving northerlies, but like ECM, had more of a north-westerly flow at T+96.

GEM scores only 5/10- northerlies at T+120 and T+96.

A rather poor set of outputs. Only ECMWF and JMA got close at T+96; GFS was reasonable at T+144 but unfortunately the GFS T+120 and T+96 outputs were among the worst.

[Thundery wintry showers]

24 June

1008mb slack low over the British Isles, with further showery rain for most, though more persistent rain was also evident in southern areas later in the day, setting us up for a decidedly dull wet 25 June in many areas.

GFS scores an 8/10- it was dodgy at T+144 with the low marginally too far SE, centred over East Anglia, but T+120 and T+96 were both very accurate.

UKMO scores only 5/10- northerlies shown at T+144 and T+120 with low pressure out in the North Sea; more accurate at T+96 but still with low pressure too far east, giving northerlies in the west.

ECMWF scores 6/10- T+144 had northerlies and low pressure in the North Sea, T+120 had the low over E Scotland, and T+96 had it over SE England. Better than UKMO but nowhere near GFS.

NOGAPS unfortunately unavailable at any timeframe.

JMA scores 7/10- low slightly too far east at T+144 and T+120 giving northerlies in central and western Britain, but very accurate at T+96.

GEM scores only 4/10, as it had the low over Scandinavia at T+120 and T+96 giving a cyclonic/north-westerly flow over the British Isles, the low must have been some 500 miles away from where it ended up.

Overall the GFS was clearly the best this time around, with the JMA surprisingly in second place. UKMO and GEM were poor and ECM not much better.

[johnholmes]

scores are, from 08/05/07

ECMWF=16

GFS=10

Met O=3

[Thundery wintry showers]

25 June

1000mb low over SE England with a decidedly cold north-east wind. East Anglia and the SE had sunshine and showers and reasonable warmth, but most other parts were dull and cold, with disastrous amounts of heavy persistent rain over Yorkshire. In view of the amount of disruption this caused, the model runs were especially critical.

Only a 5/10 for GFS I'm afraid; this wasn't a set of runs which accurately modelled the heavy rain! Low pressure well out in the North Sea at T+144 and T+120, giving a straight northerly flow, cold and cloudy perhaps but nothing spectacular. T+96 was better with low pressure off East Anglia but still had northerlies rather than north-easterlies, and the main area of rain thus probably too far east.

UKMO scores 6/10- T+144 and T+120 similar to GFS with northerlies and low pressure out in the North Sea, but T+96 was fairly accurate with low pressure over East Anglia giving NE winds; low still a little underdone though.

ECMWF also scores 6/10- it was the same as UKMO.

NOGAPS again completely unavailable.

JMA scores 7/10- T+144 was poor with northerlies and a low well out in the North Sea, but T+120 and T+96 both only slightly underdid the low, positioning it quite accurately and giving NNE winds.

GEM was poor at T+120- low over Scandinavia, NNW winds. T+96 was better, though low still underdone; overall 5/10.

If the Met Office called this event more than a few days in advance, it'll be hats off to them, because it clearly wasn't showing up in the models!

[johnholmes]

might be wrong but I think their first 'early' warning was on Friday, maybe Thursday pm; anyone got a transcript of the first one?

[Thundery wintry showers]

26 June

988mb low in the North Sea, slack 1012mb trough off W Scotland, northerlies.

GFS scores 8/10- it didn't pick out the 1012mb trough until T+96, but all three timeframes had the northerly, T+144 had the North Sea low too far east.

UKMO also scores a 8/10- poor T+144 with northerlies and HP very close by to the SW, but T+120 and T+96 were good and T+96 had the 1012mb trough- overall pretty much same as GFS.

ECMWF scores only 7/10- T+144 had low too far east, T+120 and T+96 had low accurately positioned but no 1012mb trough off W Scotland, and northerlies.

NOGAPS unavailable except T+96 when it had a cyclonic/NNW flow.

JMA scores only 6/10- North Sea low too far east at all three timeframes, with northerlies, and HP close to the SW.

GEM was okay, scoring 7/10- North Sea low quite accurately positioned but no 1012mb trough off W Scotland.

Overall a reasonable set of outputs with GFS and UKMO the best.

[Thundery wintry showers]

27 June

984mb low over southern Scandinavia, cool north-westerlies. Showers, longer spells of rain in some places.

GFS scores only 5/10- at all three timeframes low pressure was missing off N Scotland, and Britain was shown in a straight northerly flow.

UKMO scores 6/10- same problem as GFS at T+144 and T+120, but at T+96 it was starting to show the low off N Scotland, albeit underdone, giving NW winds.

ECMWF scores 7/10- T+144 was also poor with northerlies, but T+120 and T+96 had the low off N Scotland, albeit underdone, giving NW winds. T+120 was slightly closer to the mark than T+96.

NOGAPS scores 6/10- T+144 was pretty good, but T+120 and T+96 had a straight northerly rather like GFS.

JMA scores 6/10- T+144 was okay with NNW winds, T+120 was reasonably accurate with a low over N Scotland and NW winds, but T+96 then showed straight northerlies.

GEM also scores 6/10- reasonably accurate at T+120, but T+96 had Britain under straight northerlies.

Overall the models kept showing straight northerlies and not the low pressure off N Scotland (this may be why the northerlies ended up not quite as cold as first predicted).

[johnholmes]

well I'll make a final comment about my checks on these since 08/05/07

ECMWF is very clearly well ahead of GFS even after its update. So in terms of that its not really improved the GFS predictions at the time scales that Ian is running.

Scores

ECMWF=17

GFS=10

Met O=3

The rest were marked as even scores between 2 or once or twice all 3.

Edited June 28, 2007 by johnholmes

[Thundery wintry showers]

28 June

1000mb low off N Scotland giving WSW winds, moderately strong, with some rain moving east during the day. It was quite sunny over the SE for a time.

GFS was notably poor, scoring just 4/10. At T+144 it had northerlies with an Azores ridge coming into the SW, at T+120 it had a ridge of high pressure from the Azores covering most of Britain, then at T+96 it had low pressure to the north and generally north-westerly winds; the low over N Scotland was too far west.

UKMO scores 5/10; T+144 had Britain in a cyclonic/northerly flow, T+120 was quite reasonable with westerlies and an underdone low over N Scotland positioned slightly too far south, but T+96 was poor again with a cyclonic/north-westerly flow.

ECMWF scores 5/10- it was near spot on at T+144, but then drifted away from the correct outcome, showing a cyclonic/NW'ly regime at T+120 and northerlies at T+96.

NOGAPS scores 4/10- similar to GFS, T+144 and T+120 had northerlies, then T+96 showed a NW flow.

JMA also scores 4/10- low pressure over southern Britain at T+144, northerlies shown at T+120, but better at T+96 with low pressure off NE Scotland giving NW winds.

GEM scores 6/10 on the basis of T+120 and T+96- low pressure underdone and too far north with an Azores ridge into the south, but at least it had Britain in W/WSW winds at both timeframes.

A very disappointing set of outputs, with the GEM, unusually, being closest. There was a tendency to place Britain in a continued northerly regime, continuing the trend set on the previous day.

[Thundery wintry showers]

29 June

996mb low off NE Scotland, cyclonic/north-westerly flow. Heavy thundery showers affected East Anglia, further north and west it was a bright cool day with some showers in the northeast.

GFS was again poor I'm afraid, scoring just 5/10. T+144 had a strong ridge from the Azores High into southern Britain, T+120 had northerlies and high pressure to the SW, while T+96 was closer, with low pressure out in the North Sea giving northerlies, rather than north-westerlies.

UKMO scores 4/10; T+144 had an Azores ridge into southern Britain while T+120 and T+96 had a slack region of neither low nor high pressure over Britain and slack westerlies, no sign of the low off NE Scotland.

ECMWF scores 5/10- T+144 had an Azores ridge to the south giving westerlies, T+120 had slack pressure, T+96 had a weak low over the northeast, vaguely approximating to the correct pattern, so about as good as GFS overall.

NOGAPS scores 5/10- T+144 and T+120 quite poor, T+144 had northerlies and a ridge from the Azores High coming into the west, T+120 had a slack NW flow, T+96 had an underdone low over the northeast.

JMA scores 6/10; T+144 was poor with NW'ly winds and high pressure from the Azores covering the SW, but T+120 and T+96 were okay, with the low off NE Scotland too far east and NNW winds, but better than any other set of outputs for T+120/T+96.

GEM scores 5/10- T+120 was okay with the low too far north giving a slack WNW flow, but T+96 had the low missing, and a northerly flow.

Overall, a very poor set of outputs; the models really seem to be struggling at the moment. The main failing this time was to somewhat overdo a ridge extending from the Azores High, which was quite a contentious topic 4-6 days ago.

[Thundery wintry showers]

30 June

998mb low off western Ireland, with a cyclonic/southerly flow covering the British Isles. Dull wet weather spread rapidly east during the day, with some very heavy rain in places, but fortunately not much rain in the zones that were at risk of further severe flooding.

After a rather poor set of runs, GFS did better this time, scoring 8/10. T+144 and T+120 had the low slightly too far west, thus delaying the arrival of the rain, and having high pressure cling onto the east, but otherwise they were pretty good. T+96 was excellent.

UKMO was poor at T+144; low way too far west, high pressure over eastern Britain. T+120 wasn't much better; low in a more recognisable position but still high pressure covering the east, far more so than on GFS. T+96 was good, but overall it scores only 6/10.

ECMWF scores 7/10- it was excellent at T+144, but T+120 and T+96 had low pressure centred a little too far north, with high pressure over the SE and more of a SW flow over Britain.

NOGAPS scores only 5/10- T+144 and T+120 both had the low a long way too far west with a slack high over Britain, T+96 unavailable.

JMA was good at T+120 but poor at T+144 and T+96 with high pressure in charge, particularly over the south, and the low a long way west; only 5/10.

GEM scores 7/10- low too far west at T+120 but accurate at T+96.

Overall the GFS recovered from some rather poor runs earlier in the week, with ECM and GEM also reasonable. Once again the UKMO overdid the nearest area of high pressure, suggesting a more anticyclonic outlook.

[Thundery wintry showers]

1 July

996mb Low over western Britain giving a cyclonic airflow, southerly in most of the east. Most places had sun and showers, some rather more sun than others, and some rather more showers than others. Thunder was widely reported in northern England. Unfortunately I don't have the outputs for T+96.

GFS scores only 5/10- the low was well too far to the south at T+144 giving a SE flow for most, and at T+120 it still had the low too far south, with southerlies in the south and SE winds elsewhere. The implications for east-coast areas of the north (likely North Sea haar) would have been particularly large from those runs.

UKMO scores 6/10- T+144 had the low too far south, with SE winds for all, but T+120 was slightly better than the GFS, with the low too far south but southerlies over the whole of England rather than just the south.

ECMWF had low pressure too far south at T+144 with SE winds for all, but T+120 was very accurate, so 7/10 overall.

NOGAPS was good at T+144, but T+120 had the low right over southern Britain with easterlies for N England & Scotland, so scores 6/10.

JMA scores only 4/10- although T+144 was quite good with the low only slightly too far to the SW, T+120 had the low down over north-western Spain, with an east/north-easterly flow for the whole of Britain. Very poor indeed.

GEM available only at T+120 when low was too far south, giving SE winds for most.

The models generally positioned the low too far south, although the ECMWF had become spot on by T+120.

[Thundery wintry showers]

2 July

992mb low off W Scotland, with a cyclonic/westerly flow over Britain. Most parts had a mix of sun and showers and there was a fairly generous scattering of thunderstorms. As usual, some places had rather more sun and/or showers than others. T+120 outputs unavailable.

GFS scores a 7/10- quite disappointing at T+144 with low pressure centred well out in the Atlantic, though it did have Britain in westerlies, but T+96 was very accurate.

UKMO also scores 7/10- better than GFS at T+144 with a slack low over Britain, but worse than GFS at T+96; fairly accurate but with the low overdeepened and too far north.

ECMWF combined the best of GFS and UKMO and so scores 8/10- slack low over Britain at T+144, very accurate at T+96.

NOGAPS only available at T+96 when low pressure was way too far north giving a moderate WSW flow and high pressure close to the south; only 5/10.

JMA also scores only 5/10- T+144 was very poor with high pressure over the NW and NE winds in the south, while T+96 was fairly accurate but with the low underdone.

GEM only available at T+96; fairly accurate, low slightly too far west, a 7/10 overall.

The ECM was again the most consistent over the two timeframes, but GFS and UKMO were not bad.

[Thundery wintry showers]

This will be the last update until 23 or 24 July as I'm away for quite a bit. No T+144s for today.

992mb low centred off western Scotland. Many northern areas had a dull showery day, while in the south (especially East Anglia and the SE) it was a very dramatic day with some sun but also some torrential downpours with hail and numerous thunderstorms.

GFS had the main low accurately positioned, but scores only 7/10 as it positioned secondary lows over the S and SE of England that didn't materialise, at both T+120 and T+96.

UKMO likewise, thus also 7/10 for UKMO.

ECMWF was less good (only 6/10) as T+120 had the low too far east giving NW winds for all, and similar at T+96 although it was only about 100 miles too far east on that occasion.

NOGAPS scores 7/10- low overdeepened and too far east at T+120 giving strong W winds, but accurate at T+96.

JMA scores 6/10- low underdone at T+120 giving slack westerly airstream, accurate-ish at T+96 but with low slightly too far east.

GEM scores only 5/10- low too far east at both timeframes, and too far south also at T+96 with very strong westerlies over the south.

Overall a decent set of outputs but the models progged too many embedded depressions/features in the flow, which is probably why the weather has been brighter and more showery than I'd been envisaging early in the week.

[Thundery wintry showers]

These will restart on the 6th August.

[The PIT]

These will restart on the 6th August.

Glad to see you're back. Had a good time???

Looking forward to the comparisons again.

[Thundery wintry showers]

Had a pretty good time, yes- then had an event the previous weekend which made it impossible to keep updating.

5 August

Continuous records of charts started on 1 August so I only have T+96. Weak 1008mb low over Britain with SSW winds for most of eastern Britain. It was dull and wet over north-western Britain, but hot and sunny over most of England with temps of 27-30C in places.

GFS scores only 6/10 on the basis of T+96- the low was overdeepened and too far west giving strong southerlies.

UKMO scores 8/10- pretty accurate, low slightly underdone.

ECMWF scores 8/10 also- similar to UKMO, accurate, but low slightly underdone at 1012mb.

NOGAPS scores 7/10- low accurate intensity but too far south, with an easterly drift in the north.

JMA scores only 6/10- elongated low pressure area to west giving southerlies for all.

GEM had lows concentrated off W Scotland giving southerlies, only 5/10.

Overall, though T+96 is a very limited set, the Euro models handled it best at this timeframe.

[PB22]

Thanks TWS, good to see these back.

[Thundery wintry showers]

6 August

1004mb low off NE Scotland with a cyclonic/westerly flow. It was dull and wet along a frontal zone while western and southern areas were quite sunny.

GFS only scores 4/10; T+120 had a vastly overdeepened low over Ireland, while T+96 had the low vastly underdone, giving a slack northerly flow.

UKMO scores 7/10- T+120 and T+96 were both okay but had the low pressure too far south, centred over NE England.

ECMWF scores 6/10- similar to UKMO but the low too far east as well as south, giving northerlies for most.

NOGAPS scores 5/10- low over southern areas at T+120 giving easterlies, T+96 was okay but had the low too far south, centred over NE England.

JMA had the low way too far south at T+120, and way too far north at T+96, so only 3/10. Easterlies at T+120, slack pressure at T+96.

GEM scores 6/10- low over eastern Britain at T+120, low too far NE at T+96 giving north-westerlies for all.

A rather dodgy set of outputs. UKMO was the closest, while GFS was decidedly poor.

[Thundery wintry showers]

7 August

1008mb low over the far north of Scotland with a cyclonic/north-westerly flow, but slack northerlies in the south. Despite the low pressure, many parts had a dry day with sunny intervals. Heavy showers affected the extreme south.

GFS scores only 3/10. The low pressure was there, but at T+144 it was off western Ireland giving southerlies, and at T+120 and T+96 it was underdone and in the North Sea giving northerlies.

UKMO was better but still scores only 6/10. T+144 was the best output with a slightly underdone low over Scotland, but T+120 had the low in the North Sea giving northerlies, T+96 had the low in a recognisable position but slightly too far east.

ECMWF scores 3/10- low too far south at all three timeframes, with NE winds for all.

NOGAPS scores 3/10- low to the west at T+144 and T+96 giving southerlies, T+120 had the low over the North Sea giving northerlies for all.

JMA scores 5/10- T+144 had the low over SW England giving easterlies, T+120 had the low over the North Sea but T+96 was fairly accurate; low underdone but over the right area.

GEM scores just 2/10- T+120 had a ridge from the Azores High and no low pressure nearby, T+96 had the low over SE England giving north-easterlies.

Overall a poor set of runs. UKMO again was better than the others.

[Paul]

I can't recall having seen worse scores overall that that Ian! Like the new colours on the maps too

[The PIT]

Confirms my suspicions of a very short reliable time frame.

[Thundery wintry showers]

8 August

A ridge of high pressure extending over from the Azores with pressure between 1020 and 1024mb. For most, the dry and sunny but cool weather continued- an unusual combination for recent years.

GFS scores only 3/10 again. T+144 had high pressure over N Scotland, and NE winds over the south, then T+120 and T+96 had strong NE winds for all and a strong low over the SE.

UKMO scores 5/10. T+144 had a slack low over Britain, T+120 was reasonably accurate with high pressure to the SW and northerlies, but T+96 had a low to the SE and NE winds for all.

ECMWF also scores 5/10. T+144 had a low to the south giving easterlies, T+120 had a low over the top of Britain, but T+96 was by far the most accurate of any output so far. Given the very poor earlier outputs I still only give it the same as UKMO.

NOGAPS scores 4/10- lows to E and SE at T+144 and T+96 with north or northeast winds, T+120 had a low over SW England with an ESE airflow further north.

JMA scores 6/10 as it had high pressure centred too far north at all three timeframes, but elongated to the west, with northerly winds shown. Not great, but more accurate than the others.

GEM scores 5/10- T+120 had north-westerlies and a low coming into NW Scotland, while T+96 had lows to the E, giving a straight northerly flow.

Overall another poor set of outputs. Again GFS seemed particularly well out.

[The PIT]

I wonder why they're struggling at the moment. It's a bit like going back to the old days when you just knew the five day forecast was very likely to be wrong.

[Thundery wintry showers]

Some dodgy computer issues at the moment with the power supply fan thinking about terminating its services- it remains to be seen how much disruption will result. In the meantime, here's 9 August:

A region of 1020mb high pressure over Britain with 1008mb low south of Iceland. Most parts were dry, sunny and fairly cool.

GFS scores a 5/10. T+144 and T+120 had a low off SE England giving strong north/north-easterly winds over Britain, but T+96 was reasonably accurate.

UKMO scores 7/10. T+144 had the high pressure zone slightly too far west giving northerlies for the east, then T+120 and T+96 were good though T+120 had an Atlantic low too far east, 'squeezing' out the high pressure area.

ECMWF scores only 5/10- at T+120 it was reasonably accurate, but T+144 and T+96 had an Atlantic system way too far east, with SW winds for all.

NOGAPS scores 4/10- T+144 and T+96 had the high area vastly underdone with lows just to the W and E of Britain, while T+120 showed northerlies.

JMA scores 5/10- slack pressure area at T+144, at T+120 it was quite accurate, but at T+96 the Atlantic low was too far east giving SW winds.

GEM scores 6/10 as the high pressure area was correctly shown at T+120 and T+96, but vastly underdone.

Once again the UKMO came out on top.