12z Model Comparisons Results

[Thundery wintry showers]

Thursday 27th April

An anticyclonic outlook with NW winds over Britain, and a 1030mb high centred just off SW Ireland.

GFS: Well out at T+144- a strong low to the N of Scotland and strong WNW winds over Britain, the high way too far to the SW. Pretty accurate at T+120- it had an overdeepened Icelandic Low resulting in a stronger westerly flow for the far N of Scotland, but the high was correctly positioned and of the correct intensity. T+96 was excellent.

GFS still can't get above 7/10 on my marking system, and it's another 7/10 today.

UKMO: Disappointing T+144, the same errors as GFS. T+120 wasn't much better- the high too far SW, and still a cyclonic influence for Scotland, and moderate WNW winds. T+96 was excellent, though.

Only 5/10, because of the poor T+144 and T+120.

ECMWF: Had NW winds and showers at T+144, the high too far to the WSW of its correct position. At T+120 had too deep a low to the N of Scotland, with strong W winds in Scotland, but the outlook further south was quite accurate, the high a little too far SW. Accurate at T+96.

I give this 6/10, it was better than the UKMO, but not as good as GFS.

NOGAPS: Surprisingly accurate at T+144, the high only a little too far west, and a cyclonic NW flow over Britain. T+120 unavailable. T+96 unavailable.

I can't rate NOGAPS just on its T+144, which is a shame. I don't know what the people at NOGAPS are up to, but they don't seem to be producing their 12Z runs on WZ reliably.

JMA: 1010mb low over NE Scotland and slack NW winds for most at T+144, and a big Scandinavian High- couldn't possibly be more wrong. T+120- similar mistake to UKMO, too cyclonic for the north, high too far south. T+96 was similar to the ECMWF T+120- certainly getting there, but strong W winds for N Scotland.

Only 4/10.

GEM: High pressure right over Britain at T+144, the high too far east, but not a bad first stab. Way too cyclonic at T+120 with a low over Scotland and westerlies. Okay at T+96, with an anticyclonic NNW flow, and the high too far west.

The GEM scores 5/10.

Overall, a win for the GFS.

[Thundery wintry showers]

Friday 28th April

Unfortunately T+96 is unavailable.

1030mb high over Ireland, 1015mb low over SE Scandinavia, light northerlies for eastern England.

GFS: High too flat at T+144, with strong lows to the north of it, but the high was accurately positioned and gave an anticyclonic outlook for Britain. Low pressure again too strong to the north of the high at T+120- no evidence of the SE Scandinavian Low- and an anticyclonic NNW flow for Britain.

Difficult to mark this one, because it got the high pressure outlook spot on, but other details were verging on poor. Another 7/10 I feel.

UKMO: Low pressure to the NE of Scotland and unsettled NW winds at T+144. Better at T+120, but with overdeepened Scandinavian Low and a pronounced anticyclonic NNW flow.

Only 5/10.

ECMWF: Accurate at T+144- low over Scandinavia a bit far north and overdeepened, but the circulation around Britain was spot on. Very accurate at T+120, but with the flow perhaps a bit too far NW'ly, with the influence of the Russian High underdone.

Overall 8/10 for the ECMWF.

NOGAPS: Unavailable.

JMA: Had high pressure at T+144 but to the south of us, with westerly winds. Similar to UKMO at T+120, the Scandinavian Low too deep, but roughly correct high positioning.

Overall, 6/10.

GEM: Low pressure and cyclonic NW'lys at T+144. At T+120, had a pronounced Scandinavian Low, overdeepened, with the high too far west and a strong northerly flow.

Only 4/10.

So, with that, this brings my coverage up-to-date, for the first time this month! :blink:

Overall, ECMWF is still in the lead, with UKMO and GFS being equal second-best.

model_comparisons.xls

[Paul]

Thanks Ian, a bit of a posting marathon for you there!!

[Thundery wintry showers]

Saturday 29th April

We had a 1030mb Azores High centred to the SW, and slack northerlies over Britain. There was a 1010mb low, centred over the Med.

Sadly, T+120 is unavailable for this analysis. I think I accidentally left out the 24th when I was falling behind!

GFS: At T+144, it wasn't too bad. The Azores High was too far east, and the lows to the SE too far to the SE, and consequently the northerlies were underdone a little, but on the whole, the general pattern of high to the west and northerlies was correct. T+96 was superb.

Overall based on the T+144 and T+96 I give this an 8/10. This run was rather more like the sort of standard we used to expect from GFS.

UKMO: Pretty good at T+144, surprisingly for UKMO. The only real error was having the Mediterranean Low situated over Scandinavia, so our northerlies were sourced from more of a NNW point rather than NNE. T+96 was a disappointment though, with the low still situated over Scandinavia, and Britain in a north-westerly flow; although still quite good, not as good as GFS.

Overall only 7/10.

ECMWF: Pretty good again at T+144, had too significant a low pressure area to the north of the high, so our northerlies had a shorter fetch and were anticyclonic in nature, also low pressure to the SE was too far north, but overall a good stab. T+96 was superb.

Overall this was about as good as GFS, and gets 8/10.

NOGAPS: Only available at T+96. It was poor, too, with the high over western Britain, and everything too far east.

JMA: Not bad at T+144, had the low to the SE too far north, but pretty good overall. The low, and consequently the northerlies, were overdone. At T+96, a bit disappointing. The low was too close, and the winds NW'ly, and high pressure far too far out into the Atlantic.

Only 6/10.

GEM: As JMA at T+144 and again T+96, but with a little less detail. Overall T+96 was disappointing- again the low too close and too far north.

6/10 for the GEM.

A trend has emerged over the past week, with ECMWF being the most reliable of the models, but GFS now a close second, and UKMO a comparitively poor third. Whether the GFS is handling the current synoptics better than the synoptics of earlier on, or the NOAA have improved the model, or the 0.5km resolution has caused improvements, I don't know. However, ECMWF still leads comfortably in the overall standings.

model_comparisons.xls

[Thundery wintry showers]

30th April 2006

T+144 unavailable. There was a 995mb Icelandic Low, a Russian High and slack westerlies over Britain, a trough at 1016mb over the south.

GFS: Excellent T+120, everything perhaps a little too far south, but the positioning of lows and highs and the pattern over Britain was spot on. T+96 was less good- it had a continental low too far north, and consequently, easterlies over the south.

A 7/10.

UKMO: Poor T+120, overdeepened the Icelandic Low and had far too pronounced a westerly flow over Britain- too progressive (a typical GFS-like error?) At T+96. it was accurate.

A 6/10, because T+120's poor showing more than outweighed the better-than-GFS t+96.

ECMWF: Low pressure too far NE and high pressure too far North at T+120, so not very good. T+96 wasn't great either, with everything too far south, and easterlies for Scotland. However, the general pattern was close to being correct, with a slack low over southern Britain.

Only 5/10 for this one, not one of the ECMWF's greatest attempts.

NOGAPS was only available at T+96 and was poor, with a low over SE England pulling in easterlies.

JMA was too progressive at T+120 with everything too far east, and Atlantic systems pushing in, similar to UKMO but not quite as bad. T+96 was disappointing, showing easterlies.

Only 3/10, and JMA seems to have gone off the boil recently.

GEM: Too progressive at T+120 with the Atlantic low overdeepened. No T+96, unfortunately.

Overall, this was a win for the GFS, which seems to have improved lately- or is that a Murray?

[Paul]

Cheers Ian, are you onto the 0.5 degree GFS now?

[Thundery wintry showers]

1st May 2006

A rather unsettled and westerly chart, low at 995mb centred over NE Scotland and strong WNW winds over the British Isles, 990mb Icelandic Low.

GFS: Surprisingly at T+144, the GFS's failing was positioning everything too far north, and underdeepening the low (only 1005mb) over NE Scotland, though it did have us in a westerly. Poor T+120 with slack low pressure over southern Britain. T+96 and the low was too far south, at 995mb over southern England.

Only a 3/10. I must have Murrayed it, this was a poor run.

UKMO: Not bad at T+144, with a WNW flow shown over the British Isles and lows in approximately the right places. At T+120 the Icelandic Low was a bit far east, and the low over NE Scotland slightly underdone, but WNW winds correctly shown. At T+96, the main low was a bit too far SW and underdone, so not a great T+96.

Overall, a 7/10. It would have been an 8 or 9 had T+96 been as good as T+120.

ECMWF: Low pressure underdone at T+144, with slack winds over Britain and the Icelandic Low too far north. T+120: poor, with easterlies for northern Britain and a slack low over the south. T+96, poor, slack low over the south, Icelandic Low way too north.

Very uncharacteristic from the ECMWF, a very poor run that scores only 2/10. It seemed to vastly underdo the Atlantic.

NOGAPS: At T+144, main low was too far north, had westerlies over Britain, so not a bad stab. ECMWF-type error at T+120 with slack lows over southern England and easterlies in the north. No T+96.

Although there was no T+96 for NOGAPS, on the evidence of T+120 and T+144, it scores 4/10.

JMA: Good at T+144, only problem was that the main low was too far SW (same problem to UKMO T+96). Low underdone and too far south at T+120- similar to ECMWF. Same error, but worse, at T+96.

Only 3/10.

GEM: Low too far NW at T+144, but showed Britain in westerlies, lacking detail at T+120 but had us in westerlies again, the Icelandic Low positioned correctly but the other low not shown. Let down by T+96, which had a slack low over southern England and light easterlies for most.

A 5/10.

What was remarkable about this set of model outputs was that they started off with a decent output at T+144 and, in some cases, T+120, but then consistently underdid our main low and positioned it way too far south, with only the UKMO, to some extent, not committing this error. Thus, the UKMO is clearly winner of the day.

[Thundery wintry showers]

GFS outputs are indeed the new 0.5 degree ones- I think that has been the case since 27th April.

2nd May 2006

A big 970mb low off western Scotland, and strong SSW winds over Britain, 1030mb high over the Ukraine area.

GFS: T+144 had the low too far to the SE and underdeepened, resulting in a southerly rather than SSW flow, but apart from the low positioning it was good. At T+120 it had the low too far south and a secondary low at 1005mb over N Scotland, though again it had southerlies over most of Britain. T+96- almost correct, but still had the low a little too far south.

The general pattern was correct, but it consistently had the Atlantic low too far south. I say 7/10

UKMO: Low too far SE at T+144, but correctly had Britain in a southerly type regime. It was still too far SE at T+120 with a secondary low giving strong to gale force SSE winds in the SW, but the general pattern was correct. T+96 was similar to GFS- mostly accurate, but low a bit too far south.

Overall this also gets a 7/10.

ECMWF: Low way too far west at T+144 and a slack regime over Britain- overall, worse than either GFS or UKMO at T+144. The low was still too far west at T+120, but it did manage to put us in light southerlies, which is a start. At T+96 it was accurate, low only marginally too far south.

Overall I mark this down to a 5/10, because of the poor T+144 and T+120. THe ECMWF underdid the Atlantic for 1 May and seems to have done so for 2 May as well.

NOGAPS: Low underdone and over Britain at T+144, giving light SE winds for the NE, and variable winds elsewhere. T+120 wasn't too bad, but the low was too far south, similar to GFS's T+144. T+96 was similar to GFS's T+120, with a secondary low incorrectly placed over northern Britain, but a generally southerly regime.

A 5/10 for this one, not great, but not terrible either.

JMA: Low way too far SW at T+144 with only very slack southerlies for Britain. At T+120 it was still to far SW giving SE winds. Better T+96, but the low was still a bit too far SW, and certainly further so than on UKMO, GFS or ECMWF.

Only 4/10.

GEM: Low slightly underdone and too far north at T+144 giving a cyclonic SSW flow, but not too bad on the whole. Low too far south at T+120, giving southerlies, but again not bad. At T+96, it was again not too bad, with the low more or less correctly positioned, but underdone.

I give this a 7/10, it was certainly the best GEM run for a while.

So, UKMO, GFS and GEM were not outstandingly good, but they were the three models to get the pattern more or less correct at all three timeframes. The other models, including the traditionally reliable ECMWF, had problems with underdoing the Atlantic.

[Thundery wintry showers]

3rd May 2006

Low at 970mb over Iceland. 1030mb high just east of Scandinavia, with a SSW flow over Britain; high pressure holding on in the far SE.

GFS: Astonishingly accurate at T+144, with all of the lows and highs in approximately the right places, and a SSW flow for all but the SE of England (high pressure). T+120 had the low a little too far east, with the continental high therefore extending only into the far SE, but otherwise accurate. T+96 was very accurate.

Overall this run scores 9/10. It was certainly a top quality output from the GFS.

UKMO: At T+144 the Icelandic Low was displaced all the way over to western Scotland, giving a cyclonic SSW flow over Britain, the wind direction was correct but everything else was wrong. At T+120 it was quite accurate; if anything the continental flow was slightly overdone, with the low too far SW. At T+96, the low was too far east, giving a more cyclonic bias, but less so than at T+144; overall quite accurate otherwise.

This wasn't as good as the GFS, but was still quite good, so scores 7/10. The T+144- traditionally the Achilles heel of the UKMO- let the model down again.

ECMWF: At T+144 had us in a long south-easterly flow, with the main Atlantic low too far SW, and an elongated high from Scandinavia to the Med. Pretty accurate at T+120- indeed, very similar to the T+120 of GFS. At T+96 it was a top-notch output.

This was better than the UKMO, but given the error at T+144, it wasn't quite as good as the GFS, so 8/10.

NOGAPS: Actually quite good at T+144- the low too far S, giving southerlies, but the Scandinavian High was in the right place- the net effect was to have the continental airmass too far west. T+120 was a letdown though, with the low too far east, giving cyclonic SSW winds. At T+96, pretty accurate, low a bit underdone this time.

Overall a 7/10, with T+144 and T+120 both having errors, but quite a good output on the whole by the NOGAPS' standards.

JMA: Similar to ECMWF at T+144- low too far S, high to E too flat, and a SSE flow over Britain. At T+120, had the traditional JMA tendency to overdeepen the Icelandic Low and place it way too far east- just off W Scotland- giving a strong cyclonic SSW flow. Quite accurate at T+96 but still overdeepened the low, though it at least had the low in the correct place. More of a cyclonic bias to the flow over Britain.

Overall a 6/10.

GEM: Quite good T+144- low a bit far south giving southerlies, but otherwise accurate. T+120 was quite good but overdid the continental high, so we had SSE winds and a continental-origin flow for all. T+96 had the low too far SE, with a cyclonic/southerly flow for all of Britain.

Overall quite good but with errors at all three timeframes, so only 7/10.

The GFS was outstandingly accurate this time around. As I discussed in a pm conversation with Paul earlier, this improvement in the GFS recently might be related to using the 0.5 resolution output- I'm not totally sure, but it does seem that the improvement has coincided exactly with the change of resolution.

[Thundery wintry showers]

4th May 2006

So, before I go to bed tonight, the 4th May 2006. I'm not sure what's going to happen about the other thread I post in, but I'm going to make sure this one stays reasonably up-to-date, my priority will be to bring this thread up to date by the end of Thursday.

The 4th May 2006 featured a classic Spanish plume setup with SSE winds over Britain, a 990mb low centred SW of Iceland with a secondary centre west of Ireland at 995mb, and a 1010mb trough over SW England. A 1035mb high was centred over southern Scandinavia.

GFS: T+144 had the main Atlantic low centred over Iceland, so the continental flow hadn't really established on this run, with SSW winds for most, and S winds for the south-east of England. The problem was the low being too far east and north; everything else was okay. T+120 was broadly accurate, with the exception that GFS didn't pick out the 1010mb trough over SW England. At T+96 the trough was picked out, and the output was very accurate.

Mainly due to the dodgy T+144, I mark this at 7/10.

UKMO: T+144, traditionally the Achilles heel, was actually very accurate this time- the 1010mb trough over SW England wasn't picked out, but otherwise the pattern was spot on. At T+120 the low to the NW was a bit too far to the northeast, centred around Iceland, but the flow over Britain remained correct- it even had the 1010mb trough there. T+96 was accurate.

This one scores 9/10, while there were some nitpicks with low positioning, the flow over Britain was consistently correct at each timeframe.

ECMWF: A bit disappointing at T+144 with the low too far NE, and a slack easterly flow covering southern Britain, light southerlies in the north, and no low pressure to our west, just over Iceland. T+120 was the same as GFS- accurate but without the 1010mb trough. T+96 accurate, and with the 1010mb trough.

This was almost identical in performance to the GFS, so scores a 7/10.

NOGAPS: Too cyclonic at T+144- it did have southerlies, but they were rotating around a deep 970mb low south of Iceland. There was no T+120, and it was very accurate at T+96.

Overall this one scores 6/10.

JMA: Quite accurate at T+144- the low slightly too far east, so everything was displaced a little to the east, but we still had a Spanish plume over Britain. T+120 was quite good- just the continental high was a little overdone. T+96 good- but no 1010mb trough.

Overall, a 7/10.

GEM: Overdid high over Scandinavia at T+144, giving us SE winds of easterly origin. T+120- accurate, but no 1010mb trough over SW England. Same at T+96.

Overall, another 7/10.

Today, the UKMO was a clear winner, with a remarkably accurate set of outputs. The GFS has scored 7/10 or higher on seven out of the past eight days. In the overall standings, ECMWF is still in the lead, with UKMO and GFS vying for second.

model_comparisons.xls

[Thundery wintry showers]

Don't worry if everyone think that I have stopped doing these- I will be updating these tomorrow, after a hard week of work.

The other thread will probably be frozen for the time being, but this one will continue.

[Thundery wintry showers]

Updating from the 5th May on is going to be a mammoth task- is everyone okay if I start with the 11th May here, and keep updating daily from now on? (With the other thread discontinued for the time being, it shouldn't be too much work to keep this updated)

11th May 2006

There were light easterly winds over northern Scotland, and a 1020mb high off NE England, very slack winds over England and Wales. The day itself was sunny and dry for most, but scattered thundery showers broke out in places. There was a 990mb low way out in the Atlantic but largely irrelevant to Britain's weather. Northerlies trying to push into Scotland.

GFS: At T+144, had pressure slightly lower than actually happened over Europe, resulting in a pronounced easterly flow for SE England, but with a slack easterly flow shown elsewhere and the other highs and lows correctly placed. T+120 wasn't so good, everything placed a bit too far east, giving a slack SE'ly flow. T+96 quite good with high pressure at 1020mb situated over Britain.

I give this a 7/10, it was quite good, but there were errors at all three timeframes.

UKMO: The low to the SW was placed too far east at T+144, but it was quite a good stab, with pressure slack at 1015mb over Britain. As with GFS, T+120 had everything a bit too far east, with us under a slack SE'ly. At T+96 it wasn't so good, the low to the SW was too far east, enough so to give southerlies for southern Britain.

Only 6/10.

ECMWF: Had 1015mb low over SE England at T+144 giving a more pronounced easterly flow for Britain, though otherwise quite accurate. T+120 spot on, as was T+96.

I mark this one up to 8/10. T+120 and T+96 were excellent.

NOGAPS: High pressure at 1025mb too dominant at T+144, with easterlies for the south. Same error at T+120. T+96 had the low to the SW way too far east, giving southern and western Britain southerlies.

Overall only 4/10, yet another poor quality NOGAPS run.

JMA: Low to SW overdeepened and too far east at T+144, giving southerlies. T+120 still had the low too far east, but not of much consequence this time: 1020mb slack high over Britain, so pretty accurate. T+96 very good.

Overall 7/10, let down by the poor T+144.

GEM: Accurate at T+144, with 1020mb slack high over Britain and highs and lows in the right places. At T+120, had the high a little too dominant. T+96 had the low to the SW too far east, giving Britain slack southerlies.

Overall only 5/10, let down by getting worse nearer the time.

Overall, the GFS and JMA produced good runs, but the ECMWF was the winner with an excellent set of outputs, particularly T+120 and T+96.

[Thundery wintry showers]

12th May 2006

Still that low way out to the SW at 995mb not influencing our weather. A slack pressure gradient over most of Britain, with 1015mb pressure, but a NE'ly flow coming into Scotland. 1045mb Greenland High, and 1010mb Scandinavian Low. The weather was cool and cloudy for most of Scotland, but warm and humid with sun and scattered thundery showers for England & Wales & N Ireland.

GFS: At T+144 had southerlies over Britain with the low to the SW too far east, overall poor. At T+120 it was better, the low still a bit too far east, but with us under the influence of a 1020mb high, and colder NE'ly winds moving into Scotland. T+96 quite good with 1015mb slack pressure over Britain, but the low to the SW too far east, propelling southerlies into the far S of England.

Only a 6/10 for this run, it wasn't too bad, but it seemed to overdo the Atlantic low.

UKMO: It's not often I get to say this, but remarkably accurate at T+144. At T+120 it overdid the Atlantic low to the SW giving southerly winds for western Britain, but accurate further east. At T+96, accurate again.

I give this an 8/10, the T+144 and T+96 outputs were excellent, let down a little by a dodgy T+120.

ECMWF: A bit of a mess at T+144, with the pressure over and around Britain too high, but not far off overall. T+120 had the low to the SW too deep, giving a light southerly flow for Britain. T+96 was accurate.

A 7/10.

NOGAPS: High pressure too dominant at T+144- 1025mb over Britain. No T+120. T+96 had the low to the SW too far east, giving light southerlies for Britain.

Overall only 4/10.

JMA: Quite good T+144, but like ECMWF, had pressure a little too high. At T+120, it was accurate. At T+96, the low to the SW was marginally too far east, giving light southerlies for Ireland, but for England, Wales & Scotland it was another accurate run.

I give this a 7/10.

GEM: High pressure (1025mb) too dominant at T+144. High pressure to the SW too far east at T+120, giving southerlies over Britain. It was still too far east at T+96, giving southerlies, but less strong southerlies than T+96.

I only give this 4/10.

Overall, there was a consistent tendency for the models to want to position the low to the SW too far east, suggesting that the models were overdoing the progressiveness of the outlook. UKMO was, overall, just about the best of the models, with an excellent set of outputs at T+144 and T+96.

Not much change in the overall standings. ECMWF still leads, GFS and UKMO are second-best, with JMA 4th, GEM 5th and NOGAPS 6th.

model_comparisons.xls

[Thundery wintry showers]

13th May 2006

Slack pressure over Britain, ENE winds for Scotland, little wind for England and Wales, 1015mb nose of low pressure into SE England. 1000mb low, NE Scandinavia. 995mb Atlantic low, out to the SW.

For some reason my system has managed to delete the "9 May" folder without me asking it to, so it's going to be only T+120 and T+144 I'm afraid.

GFS: At T+144, had a secondary low at 1010mb off SW England that didn't actually materialise. THe main Atlantic low and Scandinavian Low were in the right places and of the right intensities, but that secondary low completely changed Britain's pattern, from a slack regime to a full-on easterly. At T+120, it had a 1010mb trough over Britain, which wasn't too accurate either.

Only 5/10 for this one.

UKMO: Atlantic low too far east at T+144, with southerlies. At T+120 the Scandinavian Low was underdone, at 1010mb, and too far west, giving northerlies. At neither timeframe did the output closely resemble the correct outcome.

Only a 3/10 for this output.

ECMWF: T+144 made mistakes with having a pronounced Euro High, and the Scandinavian Low slightly underdone. Pressure was shown at 1020mb (slack) over Britain, so at least it got Britain largely correct. Low to the SW correctly positioned. At T+120, it was spot on, the first accurate output so far.

Overall a 7/10.

NOGAPS: Atlantic low too far east at T+144, giving southerlies. The problem was still there at T+120, but it wasn't quite so far east, so eastern areas had high pressure instead. I don't know where NOGAPS got high pressure to the East from, it certainly wasn't evident on the day itself. A very poor set of outputs.

Only 2/10.

JMA: Not bad at T+144, with slack pressure at 1020mb shown over Britain, and colder ENE winds coming into Scotland, everything near-accurately positioned. T+120 even better, very accurate indeed.

Overall, this one scores 8/10.

GEM: Way wrong at T+144- it had the Atlantic low off W Scotland with a trough over N Scotland giving south-westerly winds for Britain. GEM overdid the Atlantic even more at T+120, 1010mb low over Scotland and westerlies for Britain.

Only 1/10 for this one, I even had to check to make sure the run was for the correct day, it was so bad!

A poor UKMO this time- ironic, given how good it was yesterday. Surprisingly the JMA came out with the best outputs today, while ECMWF was solid as I have come to expect recently. GFS not so hot, and NOGAPS and GEM appalling.

model_comparisons.xls

[wolves78]

so in a sence you belive the jma over the rest ATM? or is it that its seems to be the best at longer range?

[Paul]

That's just based on one date

[wolves78]

i understand that mate just trying to find out is it better ATM for long range

[Paul]

Download the excel file and it shows the results for the last couple of months

[Thundery wintry showers]

14th May 2006

Finally, the low was starting to extend from the SW, with southerlies filtering into western Britain. It was dull and wet in many parts with a front straddling the country. Weak 1020mb high over the NE, 995mb low to the SW with secondary 1000mb low forming to the east of it.

No T+120 outputs, I'm afraid.

GFS: In contrast to many recent model runs, it had the low to the SW too far west, showing Britain under slack 1020mb pressure, but the other features were correct- it had the weak frontal battleground over central areas. It was accurate at T+96.

This run scores a 7/10.

UKMO: Characteristically poor T+144, with weak high pressure to our west, and northerlies across eastern Britain, everything much further west than on the GFS. T+96 was no better, with a strong Euro High bringing westerlies to the south, and a northerly flow elsewhere, a 1010mb low over western Scandinavia, and the low to the SW way too far west.

Only 2/10 for this one.

ECMWF: T+144 was reminiscent of the GFS- everything a bit too far west, and a slack 1020mb pressure area over Britain. Accurate at T+96.

This also scores 7/10, it was almost identical to what the GFS showed.

NOGAPS: The first NOGAPS that I haven't been able to rate for a while, with only the T+96 output available. For the record, it was actually quite good- the high at 1020mb was positioned too far SE, but it did have southerlies over the west.

JMA: T+144 high pressure more dominant- overall it had the low to the SW way too far west, and a northerly flow just out in the North Sea, so marginally worse than the ECMWF and GFS outputs for T+144. Accurate at T+96.

Overall I mark this down at 6/10, because the T+144 output was worse than for ECMWF and GFS.

GEM: T+144 was poor, with low pressure over Scotland, Scandinavia and to the W, and south-westerlies shown over England & Wales. T+96, however, was accurate.

This run also scores 6/10.

In the overall standings, GFS has moved ahead of UKMO for the first time. So far this May, the ECMWF and GFS have been the most consistent performers with little to choose between them, and today's outputs reflected this.

model_comparisons.xls

[Dave J]

Appreciate the work you've put into this Ian. Makes very interesting reading.

Not surprised that ECMWF has come out top dog- I've always considered it the best model when looking at the medium range (4 days+) and this goes some way to support that. The forecasters at the Met Office take a great deal of consideration over its output often and it has a much bigger ensemble!

Very interesting also that GFS model just pips UKMet, although the difference is negligible and because the judgement is subjective it probably isn't worth reading into.

Thanks again.

--P.S. would be interesting to see the standard deviation/variance of each model (I think it's quite simple to do in Excel?) to see how "wobbly" each model is.

Edited May 17, 2006 by Dave J

[Thundery wintry showers]

15th May 2006

There was a 1000mb low off western Scotland, with southerly winds over Britain, and a deeper, 990mb, low to the SW. 1030mb high over Greenland.

T+144 unavailable.

GFS: T+120 quite good, but had the 1000mb low slightly too far south and at only 1005mb, so there were easterlies in northern Scotland; accurate with southerlies elsewhere. Similar errors at T+96- everything placed too far south, but more so than at T+120. Easterlies over the north.

Only a 6/10 for this one.

UKMO: Worse than GFS at T+120, with easterlies affecting almost the whole country, everything too far south, and to a greater extent than GFS. T+96 was even worse, with high pressure over N Scotland, a light ESE flow and the low pressure way out to the SW.

Only a 3/10.

ECMWF: Same problem at T+120- everything too far south, with easterlies for N Scotland, overall about as close as GFS got. At T+96, again, much like the GFS T+96 with everything further south still.

Same as GFS, so 6/10.

NOGAPS: High pressure in the east at T+120, so very poor again. Light southerlies shown for the W. T+96 was the same.

Only 2/10 for the NOGAPS.

JMA: Good at T+120, had high pressure a bit close by to the East, but the lows to the W correctly positioned and southerlies shown. Let down by a poor T+96 which was similar to UKMO, with high pressure over northern Scotland.

Only 5/10.

GEM: Lacking detail at T+120, but had the main weather features in the right places- southerlies across Britain and pushing northwards, lows and highs in vaguely the right places. Everything a bit far south at T+96, but closer than ECMWF and GFS.

Overall 7/10, docked a couple of marks for looking decidedly vague due to lack of detail.

Overall, it was a poor performance from the models, with GEM being the most accurate this time.

[Thundery wintry showers]

Had exams on the 18th May, so I'll update the 19th and 20th here:

19th May 2006

990mb low over N Scotland and strong westerly winds over the British Isles. 990mb low to the west of Ireland. The weather was rather cloudy and windy. There is no T+120 available this time.

GFS: T+144 was very nearly accurate, but the secondary low to the west of Ireland was placed too far east- over NW Ireland- and consequently the main rainband would have been further to the NE. T+96 was quite good, with the secondary low in the right place, but the primary low N of Scotland was positioned about 200 miles too far east.

Overall a 7/10.

UKMO: T+144 was poor with both lows positioned a long way too far west, though it did have a westerly regime covering Britain. T+96 underdid the low to the west of Ireland (only 990mb) and overdid the low over N Scotland giving stronger W winds, though the lows and highs were in the right places.

Only 6/10 for the UKMO.

ECMWF: Had the low over N Scotland about 100 miles too far SW at T+144, and the Atlantic low way too far SW, though westerlies were correctly shown over Britain. T+96 was better, but the low in the Atlantic was rather underdone and too far south.

Overall, 6/10.

NOGAPS: Unfortunately I only have the T+96 for the NOGAPS, and for the record, it was poor with both lows way too far east.

JMA: Very poor T+144- the main low centred over southern Britain, and the Atlantic low nowhere to be seen. At T+96 the low over N Scotland was correctly placed but the secondary low way too far east, over N Ireland, so Britain was showing as having south-westerlies.

Only 4/10.

GEM: Main low to N of Scotland about 500 miles too far east at T+144, and the secondary low way too far west, giving northerlies for northern Britain. T+96, and the main low N of Scotland was accurately predicted, but the secondary low still way too far west.

Only a 4/10.

Another quite poor day for the models, GFS winning out because of its reasonable T+144.

To compensate for being a bit lax in updating these recently, I have decided, upon Dave J's suggestion, to include the standard deviations of the models. All of them are quite similar, but UKMO has the lowest standard deviation, followed by ECMWF and then GFS.

model_comparisons.xls

[Thundery wintry showers]

20th May 2006

A 990mb low just off eastern England, another off NE Scotland, and northerlies over Britain, 990mb low also to the SW, waiting to stream in off the Atlantic.

No T+144 I'm afraid.

GFS: Mostly accurate at T+120, but missing the low off NE Scotland, everything else was accurately positioned, so the weather for England and Wales was accurate, but it showed a more pronounced NE flow for Scotland. Good also at T+96, low off E England about 100 miles further east of the correct position but everything else spot on.

Quite a good run, but some errors. I give it 8/10.

UKMO: Poor T+120, with no low in the Atlantic, and the main lows over the British Isles rather than to the east. However, T+96 was accurate, the lows more or less correctly placed, the low off NE Scotland underdone though.

Only 6/10.

ECMWF: T+120 was corrupted, so unavailable. It's probably just as well for the people at ECMWF that I couldn't mark this one, because T+96 was poor- the main low off E England was overdeepened and positioned over central southern England, so it would at least have been worse than GFS.

NOGAPS: No data.

JMA: At T+120, the low off NE Scotland was positioned over N Scandinavia, and the low off E England was underdone and positioned too far north. At T+96, similar to the GFS T+120, but with the low off E England too far west, situated over eastern England, and missing the low off NE Scotland.

Only 5/10.

GEM: T+120 was unavailable so I can't give this a mark, but T+96 was poor with the low off E England vastly underdone.

Overall, a good day for the GFS. Is the GFS returning to its former glory or is this just a blip?

[Thundery wintry showers]

21st May 2006

985mb low to the north of Scotland. 980mb low to the west of SW England. Southerlies over England and Wales, easterlies for Scotland.

T+96 unavailable.

GFS: At T+144, the low to the north of Scotland was missing, but the low off SW England was of the correct intensity, and placed quite accurately, about 100 miles south of where it ended up. The main rain band would have ended up further south as a result. At T+120, still no sign of the low to the north of Scotland, but the low off SW England was correctly positioned and of the correct intensity, so for Britain a very accurate projection.

I give this 7/10- where did that low north of Scotland disappear to? Otherwise good.

UKMO: The trademark poor T+144 was intact- no sign of any low off SW England, and a 985mb low over NW Scotland, resulting in moderate westerlies for most, strong to gale force in NW Scotland. T+120 was better, the low off SW ENgland about 100 miles too far south but otherwise accurately placed, and the low off N Scotland was there, but placed over Scandinavia, quite a way east of where it ended up.

Overall only 5/10.

ECMWF: T+144 as GFS, the low off SW England too far south, and the low off N Scotland non-existent, but otherwise a fairly accurate run. T+120 also accurate, again no low off N Scotland.

This gets 7/10. It's astonishing me how the ECMWF and GFS have been backing each other up a lot recently.

NOGAPS: At T+144 had a Scandinavian Low, no low off SW England, and north-westerlies. T+120 had 980-985mb lows off NE Scotland and strong north-westerlies.

Only 2/10 I'm afraid.

JMA: Had northerlies at T+144. The low off SW England was there, but positioned way too far west. At T+120, quite accurate, the low to the SW slightly overdeepened and too far SW but only by about 100 miles, the low off N Scotland was however placed over Scandinavia.

Only 5/10.

GEM: Only T+120 available so I can't mark it, but T+120 had easterlies over Britain, and the low to the SW way underdone and too far south.

Overall, the ECMWF and GFS were the day's best models, being remarkably similar in their outputs.

[Thundery wintry showers]

22nd May 2006

A big 990mb low over SE England, with resultant wind and rain and a lot of cloud. For northern areas there were north-easterly winds. 985mb low north of Scotland, 1000mb low south of Iceland, 1025mb high way out in the Azores.

This time we don't have T+120; 17 May was the day before my exam in Weather and Forecasting.

GFS: At T+144 the low over SE England was remarkably accurate both in positioning and intensity, however the other two lows I mentioned were not present, and pressure was high to the west generally. As a result strong ENE winds were shown for the north- so accurate for England & Wales, but not so much so for elsewhere. T+96 also not bad, had the low over SE England positioned too far NE, sat off the Lincolnshire coast, but everything else was pretty good.

Some issues, but a moderately impressive GFS run, another 7/10 I think.

UKMO: Remarkably accurate at T+144, the low over SE England accurately placed and of the right intensity- it even had the low north of Scotland correctly placed. Unfortunately it ruined it with a poor T+96, with the low over SE England placed over NE Scotland, and strong north-westerly winds shown for Britain.

I give this 6/10; the T+144 was so accurate that I award more than half-marks, but really, the T+96 was very poor and certainly much worse than the GFS.

ECMWF: At T+144 the low over SE England was positioned off the coast of SW England, so a bit too far west, and also overdeepened at 980mb. Other features were accurately placed. Winds over Britain were shown as easterly. At T+96, it was as accurate as it is possible to get- it's hard to spot any differences between the actual outcome and what the ECMWF showed at T+96.

This one also scores 7/10; the T+96 was better than the GFS, but the T+144 was worse.

NOGAPS: No low over SE England at T+144, lows north of Scotland, westerlies. Not good. At T+96, it was close, but the low over SE England was placed just off the coast of SE England, i.e. a bit too far east.

Overall 5/10. At least T+96 was quite close.

JMA: T+144 was impressive, about as much so as UKMO, with the main low only slightly too far W, and the other lows accurately placed also. T+96 also superb, the only quibble being the lack of a 1000mb low out to the NW, everything else very accurately placed.

Overall this scores a 9/10.

GEM: Similar to ECMWF at T+144 with the SE ENgland low placed too far W. The T+96 was however much worse than the ECMWF, with no sign of a low over SE England, and instead a deep low over N Scotland giving strong cyclonic westerly winds, so a bit of a write-off.

Overall, just 4/10.

A surprising win for the JMA model this time, with a very accurate set of outputs. Meanwhile, in the overall standings ECMWF is clear first, and GFS is establishing itself as clear second ahead of the UKMO.

model_comparisons.xls