Jump to content
Snow?
Local
Radar
Cold?
IGNORED

Call For Forecasts -- Predict 20 March Before 4 March


Recommended Posts

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Day six -- can you believe it? The trend on the models is very slightly towards weaker high pressure but still over the same general region from southern England east into the Baltic and Russia. Some other runs of the GFS have shown considerably more erosion than the 12z continues to advertise, and now the UKMO and GEM are out to 144h and both look quite similar to the ECM, which in general has continued a trend of weakening the high and suggesting more of a trough developing from 30W towards 20W near forecast time. This is by no means a powerhouse feature but it stands in contrast to strong high pressure that had been over the same general area on some model runs previously.

In other words, the models seem to be picking up some of the energy discussed in this thread, but without any strong signs of development at this rather late stage.

Since neither model has changed greatly over 24h, the correlations remain similar. First of all, the ECM correlates at .96 with its previous day's effort, and the GFS at .90 ... and the inter-model correlation has risen to 0.71.

Now, here's today's updated table of the thirteen maps being correlated, and the NW ensemble values.

Fcst ... day 10 ....... day 9 ....... day 8 ...... day 7 ...... day 6 (all days GFS (ECM))

ICE ..... .29 (.44) ... .81 (.33) .... .53 (.70) ... .57 (.82) ... .54 (.84)

SFT .... -.07 (.36) ... .51 (.45) ... .61 (.96) ... .39 (.91) ... .37 (.83)

GP2 .... -.06 (.29) ... .53 (.46) ... .61 (.94) ... .42 (.87) ... .38 (.83)

GP1 .... .05 (.21) ... .59 (.47) .... .54 (.82) ... .50 (.77) ... .37 (.81)

MB. .... -.13 (.45) ... .64 (.43) ... .47 (.80) ... .39 (.82) ... .44 (.77)

NW. .... -.18 (.21) ... .44 (.61) ... .38 (.90) ... .24 (.80) ... .18 (.73)

PB. .... -.10 (.39) ... .47 (.07) ... .78 (.73) ... .51 (.75) ... .54 (.70)

RIC .... -.16 (-.07) ... .44 (.39) ... .30 (.65) ... .31 (.59) ... .16 (.60)

BLU ..... .80 (.52) ... .65 (.18) ... .19 (.14) ... .56 (.39) ... .51 (.48)

RJS .... -.26 (-.11) ... .03 (.43) ... -.03 (.39) ... .05 (.32) ... -.15 (.30)

STM .... -.66 (-.09) .. -.15 (.52) .. -.03 (.64) ... -.36 (.37) ... -.34 (.20)

TWS .... .51 (.36) ... .38 (-.54) .. .34 (-.32) ... .48 (-.05) ... .52 (.11)

COA ... -.02 (-.22) ... -.04 (.66) .. -.40 (.12) .. -.23 (-.06) .. -.35 (-.03)

LOM ... -.68 (-.29) ... -.69 (.50) .. -.49 (.12) .. -.86 (-.24) .. -.72 (-.39)

The order would change considerably if the GFS was our basis ... the top five GFS correlations are

PB .54

ICE .54

TWS .52

BLU .51

MB .44

The next few days will provide an interesting test of the energy level theory, since this energy peak from the 19-21 March events would begin to show up increasingly in the data given the 7-10 day energy cycle involved. There is considerable divergence in the ECM and GFS solutions past 21 March, suggesting that the models have some potential to change over this 5-6 day time period. (The GFS has a much colder solution for the week after 21 March)

Link to comment
Share on other sites

In other words, the models seem to be picking up some of the energy discussed in this thread, but without any strong signs of development at this rather late stage.

You are correct, the energy is there. Its just a question of whether it manifests itself in the part of the world covered by the forecast map on 20 March 2011.

The events around the world, in all aspects of Nature, over the last two and a half months suggest this will be an "extreme" kind of year.

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

I have done the GFS correlations for today, no big changes there as the map correlates at .86 with the previous day's output. I notice the GEM today looks similar now. Posting this note because I have to go out for part of the day here and won't see the ECM until I get back, so anyone looking for the update, check back around 9-10 p.m. your time.

I notice on the GEM 120h map that there is an area of vorticity embedded in the southwest flow to the west of Ireland, that will probably form some kind of frontal wave if not a separate low. I continue to think there is some potential for greater development in areas north and west of the UK and Ireland but the models hold firm so far to a rather bland solution.

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Now catching up with the ECM data, this also remains static with correlation of .87 to previous day. The two models now correlate at .82, beginning to converge on the final answer to this puzzle.

Or will there be any last minute shifts to help some of our struggling entries? Some would benefit from a weaker high and more isobars between the central UK and Iceland. Others would be helped by the opposite trend.

As of today, then, the race to the finish looks like this (based on ECM but with GFS notables listed after the main table) :

Fcst ... day 10 ....... day 9 ....... day 8 ...... day 7 ...... day 6 ...... day 5 (all days GFS (ECM))

MB. .... -.13 (.45) ... .64 (.43) ... .47 (.80) ... .39 (.82) ... .44 (.77) ... .70 (.83)

ICE ..... .29 (.44) ... .81 (.33) .... .53 (.70) ... .57 (.82) ... .54 (.84) ... .79 (.82)

SFT .... -.07 (.36) ... .51 (.45) ... .61 (.96) ... .39 (.91) ... .37 (.83) ... .71 (.81)

GP2 .... -.06 (.29) ... .53 (.46) ... .61 (.94) ... .42 (.87) ... .38 (.83) ... .70 (.81)

NW. .... -.18 (.21) ... .44 (.61) ... .38 (.90) ... .24 (.80) ... .18 (.73) ... .57 (.73)

GP1 .... .05 (.21) ... .59 (.47) .... .54 (.82) ... .50 (.77) ... .37 (.81) ... .67 (.68)

PB. .... -.10 (.39) ... .47 (.07) ... .78 (.73) ... .51 (.75) ... .54 (.70) ... .64 (.64)

RIC .... -.16 (-.07) ... .44 (.39) ... .30 (.65) ... .31 (.59) ... .16 (.60) ... .46 (.55)

STM .... -.66 (-.09) .. -.15 (.52) .. -.03 (.64) ... -.36 (.37) ... -.34 (.20) ... .00 (.39)

BLU ..... .80 (.52) ... .65 (.18) ... .19 (.14) ... .56 (.39) ... .51 (.48) ... .59 (.36)

RJS .... -.26 (-.11) ... .03 (.43) ... -.03 (.39) ... .05 (.32) ... -.15 (.30) ... .14 (.09)

COA ... -.02 (-.22) ... -.04 (.66) .. -.40 (.12) .. -.23 (-.06) .. -.35 (-.03) ... .13 (.06)

TWS .... .51 (.36) ... .38 (-.54) .. .34 (-.32) ... .48 (-.05) ... .52 (.11) ... .25 (-.03)

LOM ... -.68 (-.29) ... -.69 (.50) .. -.49 (.12) .. -.86 (-.24) .. -.72 (-.39) ... -.56 (-.09)

The order would change considerably if the GFS was our basis ... the top sevem GFS correlations are

ICE .79

SFT .71

MB .70

GP2 .70

GP1 .67

PB .64

BLU .59

TWS .52

I've been tracking the European high on the models and finding this trend: mainly a gradual weakening and after some bobbing east and west (actually ENE and WSW) of the current predicted position (southern North Sea) a gradual "homing in" on this location. Looking at the various forecast maps, the closest estimates considering position, intensity and flow, to this European high (currently near Holland 1037 mbs) would seem to be these:

Murcie Boy 1037 western France (error 10 deg and 0 mb)

Iceberg 1028 Biscay (error 15 deg and 9 mb)

Paul B 1043 Poland (error 10 deg and 6 mb)

RJS 1032 Poland (error 10 deg and 5 mb)

Many others in the "NW ensemble" were reasonably close on intensity but too far west or north (substantially altering the flow pattern) and the NW ensemble itself lags behind these four on this feature. The NW ensemble would read

NW 1028 western Biscay (error 20 deg and 9 mb)

The Icelandic low which has come and gone from the models on a quasi-diurnal cycle seems to be settling on a solution near western Iceland but this is rather uncertain, and the two-low solution that you see on my map is hinted at although on a wider scale over the eastern half of the Atlantic. It will be worth plotting close estimates to either the Icelandic low or any low further west in the grid, if we get better results near the end of the journey. Otherwise, the solutions without a defined low in the grid are perhaps closest.

BTW, looks like Murcie Boy has done a lot better with this challenge than the 1-5 Feb business, wouldn't you say?

Edited by Roger J Smith
Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Perhaps the unexpected result here is that the NW ensemble is a pretty powerful 30-day forecast tool if this is any indication. I wonder if we should look at some sort of weighted ensemble that we do each month around the same time to get a mid-month estimate of conditions next month? By weighted, I mean weighting the participants by previous accuracy on a flexible basis. I'm also looking at the idea of taking the GP maps and using them as flow pattern estimates for positioning events in the energy level model. That would have worked quite well in this case.

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Day four correlations

________________________________

Well, almost there now, the models are locking into a solution with .94 correlation for today's GFS and ECM, and from visual inspection, about the same maps presented by GEM and UKMO. All now have a slight trough embedded in the southwest flow near western Ireland, perhaps an area of upper level cloud and a few showers to represent what I had expected might be a developing low. Or will there be last minute shifts in these maps? We shall see.

The GFS meanwhile correlated at .96 to its previous day's 12z effort, and the ECM at .90 to its previous map. The ECM shows higher correlations to days 6 and 7 (.94, .96). The GFS correlations drop off fairly smoothly then oscillate as you go back into FI, the best long-shot correlation being .84 with day 12, and the worst .16 with day 14.

Here is today's roundup for our 13 NW entrants and the NW ensemble:

Fcst ... day 10 ....... day 9 ....... day 8 ...... day 7 ...... day 6 ...... day 5 ...... day 4 (all days GFS (ECM))

SFT .... -.07 (.36) ... .51 (.45) ... .61 (.96) ... .39 (.91) ... .37 (.83) ... .71 (.81) ... .75 (.86)

GP2 .... -.06 (.29) ... .53 (.46) ... .61 (.94) ... .42 (.87) ... .38 (.83) ... .70 (.81) ... .72 (.83)

MB. .... -.13 (.45) ... .64 (.43) ... .47 (.80) ... .39 (.82) ... .44 (.77) ... .70 (.83) ... .68 (.77)

ICE ..... .29 (.44) ... .81 (.33) .... .53 (.70) ... .57 (.82) ... .54 (.84) ... .79 (.82) ... .72 (.76)

PB. .... -.10 (.39) ... .47 (.07) ... .78 (.73) ... .51 (.75) ... .54 (.70) ... .64 (.64) ... .76 (.75)

GP1 .... .05 (.21) ... .59 (.47) .... .54 (.82) ... .50 (.77) ... .37 (.81) ... .67 (.68) ... .64 (.72)

NW. .... -.18 (.21) ... .44 (.61) ... .38 (.90) ... .24 (.80) ... .18 (.73) ... .57 (.73) ... .58 (.69)

RIC .... -.16 (-.07) ... .44 (.39) ... .30 (.65) ... .31 (.59) ... .16 (.60) ... .46 (.55) ... .44 (.49)

BLU ..... .80 (.52) ... .65 (.18) ... .19 (.14) ... .56 (.39) ... .51 (.48) ... .59 (.36) ... .48 (.38)

STM .... -.66 (-.09) .. -.15 (.52) .. -.03 (.64) ... -.36 (.37) ... -.34 (.20) ... .00 (.39) ... .05 (.26)

RJS .... -.26 (-.11) ... .03 (.43) ... -.03 (.39) ... .05 (.32) ... -.15 (.30) ... .14 (.09) ... .14 (.19)

TWS .... .51 (.36) ... .38 (-.54) .. .34 (-.32) ... .48 (-.05) ... .52 (.11) ... .25 (-.03) .. .28 (.04)

COA ... -.02 (-.22) ... -.04 (.66) .. -.40 (.12) .. -.23 (-.06) .. -.35 (-.03) ... .13 (.06) .. -.18 (-.01)

LOM ... -.68 (-.29) ... -.69 (.50) .. -.49 (.12) .. -.86 (-.24) .. -.72 (-.39) ... -.56 (-.09) .. -.46 (-.29)

The order would change if the GFS was our basis ... the top eight GFS correlations are

PB .76

SFT .75

ICE .72

GP2 .72

MB .68

GP1 .64

BLU .48

RIC .44

The European high remains about the same as discussed yesterday, but the Icelandic low is somewhat "back" on the map to the east of Iceland.

We should note for the record that four-day prog charts can develop significant errors and this is not quite a done deal yet.

More tomorrow ...

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

Getting into MetO Fax chart territory now:

PPVM89.png

I've got the high in the North Sea (albeit wildly out!) and some pressure centres are again a bit closer (but completely wrong). However, on the positive side, everything else seems a million miles away! :rofl:

post-6667-0-06078400-1300353758.jpg

post-6667-0-06078400-1300353758_thumb.jp

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Here now are the results for the 72h forecasts, the day 3 validation.

The maps are now settling into a nearly repetitive pattern and the GFS correlates with its previous effort at .95, the ECM at .94, and the inter-model correlation is .95.

If anything, there was a slight drift towards a weaker high and a stronger southwest flow to its northwest, but those were very subtle changes. They are helping my map slightly because I tended to have the biggest disturbance in the flow to the west or northwest of Ireland. Many correlations have edged upwards from yesterday because of that stronger gradient. The new list looks like this:

Fcst ..... day 10 ...... day 9 ...... day 8 ..... day 7 ..... day 6 ..... day 5 ..... day 4 ..... day 3 (all days GFS (ECM))

SFT .... -.07 (.36) .. .51 (.45) .. .61 (.96) .. .39 (.91) .. .37 (.83) .. .71 (.81) .. .75 (.86) .. .86 (.90)

GP2 .... -.06 (.29) .. .53 (.46) .. .61 (.94) .. .42 (.87) .. .38 (.83) .. .70 (.81) .. .72 (.83) .. .83 (.88)

NW. .... -.18 (.21) .. .44 (.61) .. .38 (.90) .. .24 (.80) .. .18 (.73) .. .57 (.73) .. .58 (.69) .. .69 (.80)

ICE ..... .29 (.44) .. .81 (.33) ... .53 (.70) .. .57 (.82) .. .54 (.84) .. .79 (.82) .. .72 (.76) .. .76 (.78)

PB. .... -.10 (.39) .. .47 (.07) .. .78 (.73) .. .51 (.75) .. .54 (.70) .. .64 (.64) .. .76 (.75) .. .75 (.78)

GP1 .... .05 (.21) .. .59 (.47) ... .54 (.82) .. .50 (.77) .. .37 (.81) .. .67 (.68) .. .64 (.72) .. .71 (.78)

MB. .... -.13 (.45) .. .64 (.43) .. .47 (.80) .. .39 (.82) .. .44 (.77) .. .70 (.83) .. .68 (.77) .. .75 (.77)

RIC .... -.16 (-.07) .. .44 (.39) .. .30 (.65) .. .31 (.59) .. .16 (.60) .. .46 (.55) .. .44 (.49) .. .51 (.68)

RJS .... -.26 (-.11) .. .03 (.43) .. -.03 (.39) .. .05 (.32) .. -.15 (.30) .. .14 (.09) .. .14 (.19) .. .22 (.36)

BLU ..... .80 (.52) .. .65 (.18) .. .19 (.14) .. .56 (.39) .. .51 (.48) .. .59 (.36) .. .48 (.38) .. .41 (.35)

STM .... -.66 (-.09) . -.15 (.52) .. -.03 (.64) . -.36 (.37) . -.34 (.20) .. .00 (.39) .. .05 (.26) .. .24 (.30)

TWS .... .51 (.36) .. .38 (-.54) .. .34 (-.32) .. .48 (-.05) .. .52 (.11) .. .25 (-.03) .. .28 (.04) .. .04 (.02)

COA ... -.02 (-.22) . -.04 (.66) . -.40 (.12) . -.23 (-.06) . -.35 (-.03) .. .13 (.06) .. -.18 (-.01) .. -.14 (-.02)

LOM ... -.68 (-.29) . -.69 (.50) . -.49 (.12) . -.86 (-.24) . -.72 (-.39) . -.56 (-.09) .. -.46 (-.29) .. -.33 (-.31)

The GFS top correlations are

SFT .86

GP2 .83

ICE .76

PB. .75

MB. .75

GP1 .71

Back again tomorrow with more of this, seems that we have quite a crop of savants here.

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

Not looking much better for me, but it's starting to prove something for this forecast which I will highlight after Sunday!

post-6667-0-63157800-1300442190.png

post-6667-0-63157800-1300442190_thumb.pn

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

48h (day 2) validation

_______________________________

The trend continues towards a weaker European high, stronger southwest flow in the near Atlantic north and west of Ireland and Scotland, and today, a return to a more defined Icelandic low near the western boundary of the validation grid although not the map grid that we all drew our entries on.

What it means is further upward drift of the correlation scores, a sort of faint signal of the "raging zonality" that one entrant spoke about, and if you check the UKMO there is an even stronger trend in this direction, the pressure at Shetland on that model is closer to 1010 mb than the 1020s on the two main models.

On the other hand the maps today have only changed marginally for correlation purposes with previous days, and the inter-model correlation is .99 while the GFS checks in at .96 with its previous day, and the ECM at .97.

The correlation for the map submissions looks like this today, but I have listed them in order of average correlation since the two models are converging on the same solution:

Fcst ..... day 10 ...... day 9 ...... day 8 ..... day 7 ..... day 6 ..... day 5 ..... day 4 ..... day 3 ..... day 2 (all days GFS (ECM))

SFT .... -.07 (.36) .. .51 (.45) .. .61 (.96) .. .39 (.91) .. .37 (.83) .. .71 (.81) .. .75 (.86) .. .86 (.90) ... .91 (.91)

GP2 .... -.06 (.29) .. .53 (.46) .. .61 (.94) .. .42 (.87) .. .38 (.83) .. .70 (.81) .. .72 (.83) .. .83 (.88) ... .90 (.91)

NW. .... -.18 (.21) .. .44 (.61) .. .38 (.90) .. .24 (.80) .. .18 (.73) .. .57 (.73) .. .58 (.69) .. .69 (.80) ... .80 (.80)

PB. .... -.10 (.39) .. .47 (.07) .. .78 (.73) .. .51 (.75) .. .54 (.70) .. .64 (.64) .. .76 (.75) .. .75 (.78) ... .78 (.81)

ICE ..... .29 (.44) .. .81 (.33) ... .53 (.70) .. .57 (.82) .. .54 (.84) .. .79 (.82) .. .72 (.76) .. .76 (.78) ... .80 (.78)

MB. .... -.13 (.45) .. .64 (.43) .. .47 (.80) .. .39 (.82) .. .44 (.77) .. .70 (.83) .. .68 (.77) .. .75 (.77) ... .79 (.79)

GP1 .... .05 (.21) .. .59 (.47) ... .54 (.82) .. .50 (.77) .. .37 (.81) .. .67 (.68) .. .64 (.72) .. .71 (.78) ... .79 (.78)

RIC .... -.16 (-.07) .. .44 (.39) .. .30 (.65) .. .31 (.59) .. .16 (.60) .. .46 (.55) .. .44 (.49) .. .51 (.68) ... .66 (.64)

BLU ..... .80 (.52) .. .65 (.18) .. .19 (.14) .. .56 (.39) .. .51 (.48) .. .59 (.36) .. .48 (.38) .. .41 (.35) ... .39 (.34)

STM .... -.66 (-.09) . -.15 (.52) .. -.03 (.64) . -.36 (.37) . -.34 (.20) .. .00 (.39) .. .05 (.26) .. .24 (.30) ... .32 (.34)

RJS .... -.26 (-.11) .. .03 (.43) .. -.03 (.39) .. .05 (.32) .. -.15 (.30) .. .14 (.09) .. .14 (.19) .. .22 (.36) ... .32 (.28)

COA ... -.02 (-.22) . -.04 (.66) . -.40 (.12) . -.23 (-.06) . -.35 (-.03) .. .13 (.06) .. -.18 (-.01) .. -.14 (-.02) ... .02 (.01)

TWS .... .51 (.36) .. .38 (-.54) .. .34 (-.32) .. .48 (-.05) .. .52 (.11) .. .25 (-.03) .. .28 (.04) .. .04 (.02) ... .00 (-.02)

LOM ... -.68 (-.29) . -.69 (.50) . -.49 (.12) . -.86 (-.24) . -.72 (-.39) . -.56 (-.09) .. -.46 (-.29) .. -.33 (-.31) ... -.28 (-.24)

Although the two leading entries have scored very well, they are both about equally "off the mark" in terms of intensity, one being too intense and the other (predictably as an index sort of model forecast) is not intense enough. Using either for wind speed forecasts would result in some moderately large errors but directions would be good and from my own subjective interpretation the temperature fields are good. This is also the case for some of the pack of forecasts that follow.

The large group just behind the NW consensus are all virtually tied with that consensus as far as the range of uncertainty goes, and then we have some other forecasts that have less robust correlation but still caught some of the pattern. Some maps like that of MB have a good combination of position and intensity but fall slightly short on the exact positions so they lose some correlation value there. Others (like mine) have more intensity issues than frontal placement problems.

At least that's the story from the 48 hour model charts, the story at game time could be tweaked a little in one direction or another.

I've got to say the NW ensemble correlations are way higher than I would have wagered a month ago, .80 is pretty much the theoretical upper limit for this sort of exercise. We could do this every day for a year and probably not exceed that value more than two or three times. The value at random would probably be closer to 0.20. As a consequence, we have far higher individual correlations than you would normally find in this sort of challenge.

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

A little over 24 hours to go and I've found this very interesting. Who will emerge with the best guess?

post-6667-0-31918300-1300528516.jpg

post-6667-0-31918300-1300528516_thumb.jp

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

24h (day one) validation

___________________________________

The 24h maps both correlate at .98 with their previous day's efforts and with each other (GFS and ECM) so the various correlations for our thirteen maps and the NW ensemble remain just about the same as yesterday. Here's the updated table of stats, order determined by average of the two correlations (as with yesterday) ...

Fcst ..... day 10 ...... day 9 ...... day 8 ..... day 7 ..... day 6 ..... day 5 ..... day 4 ..... day 3 ..... day 2 ... day 1

................................... (all days GFS (ECM)) ...........................

SFT .... -.07 (.36) .. .51 (.45) .. .61 (.96) .. .39 (.91) .. .37 (.83) .. .71 (.81) .. .75 (.86) .. .86 (.90) ... .91 (.91) ... .92 (.92)

GP2 .... -.06 (.29) .. .53 (.46) .. .61 (.94) .. .42 (.87) .. .38 (.83) .. .70 (.81) .. .72 (.83) .. .83 (.88) ... .90 (.91) ... .91 (.92)

PB. .... -.10 (.39) .. .47 (.07) .. .78 (.73) .. .51 (.75) .. .54 (.70) .. .64 (.64) .. .76 (.75) .. .75 (.78) ... .78 (.81) ... .81 (.83)

NW. .... -.18 (.21) .. .44 (.61) .. .38 (.90) .. .24 (.80) .. .18 (.73) .. .57 (.73) .. .58 (.69) .. .69 (.80) ... .80 (.80) ... .80 (.82)

GP1 .... .05 (.21) .. .59 (.47) ... .54 (.82) .. .50 (.77) .. .37 (.81) .. .67 (.68) .. .64 (.72) .. .71 (.78) ... .79 (.78) ... .77 (.78)

MB. .... -.13 (.45) .. .64 (.43) .. .47 (.80) .. .39 (.82) .. .44 (.77) .. .70 (.83) .. .68 (.77) .. .75 (.77) ... .79 (.79) ... .77 (.77)

ICE ..... .29 (.44) .. .81 (.33) ... .53 (.70) .. .57 (.82) .. .54 (.84) .. .79 (.82) .. .72 (.76) .. .76 (.78) ... .80 (.78) ... .72 (.73)

RIC .... -.16 (-.07) .. .44 (.39) .. .30 (.65) .. .31 (.59) .. .16 (.60) .. .46 (.55) .. .44 (.49) .. .51 (.68) ... .66 (.64) ... .60 (.65)

STM .... -.66 (-.09) . -.15 (.52) .. -.03 (.64) . -.36 (.37) . -.34 (.20) .. .00 (.39) .. .05 (.26) .. .24 (.30) ... .32 (.34) ... .39 (.40)

RJS .... -.26 (-.11) .. .03 (.43) .. -.03 (.39) .. .05 (.32) .. -.15 (.30) .. .14 (.09) .. .14 (.19) .. .22 (.36) ... .32 (.28) ... .34 (.34)

BLU ..... .80 (.52) .. .65 (.18) .. .19 (.14) .. .56 (.39) .. .51 (.48) .. .59 (.36) .. .48 (.38) .. .41 (.35) ... .39 (.34) ... .30 (.28)

COA ... -.02 (-.22) . -.04 (.66) . -.40 (.12) . -.23 (-.06) . -.35 (-.03) .. .13 (.06) .. -.18 (-.01) .. -.14 (-.02) ... .02 (.01) ... .03 (.01)

TWS .... .51 (.36) .. .38 (-.54) .. .34 (-.32) .. .48 (-.05) .. .52 (.11) .. .25 (-.03) .. .28 (.04) .. .04 (.02) .. .00 (-.02) . -.07 (.07)

LOM ... -.68 (-.29) . -.69 (.50) . -.49 (.12) . -.86 (-.24) . -.72 (-.39) . -.56 (-.09) . -.46 (-.29) . -.33 (-.31) . -.28 (-.24) . -.19 (-.15)

Now this is interesting too, the change in correlation (ECM only as GFS was poor then) from day 10 to day 1 looks like this in the same order:

SFT +.56

GP2 +.63

NW +.61

GP1 +.57

MB +.32

ICE +.19

RIC +.72

STM +.49

RJS +.45

BLU -.24

COA +.23

TWS -.29

LOM +.14

Most of us have gained correlation over the nine days and the greatest identified reason for this would be increasing zonality and development of lows at the expense of the high. Maps losing ground had started off with more high pressure in the western parts of the grid.

Unless the supermoon scrambles up the pressure grid in a major way overnight, we can expect a very similar outcome at validation time. I'll provide an "actual" map for reference with fronts and pressures on the same base map as we were using.

Plenty of good efforts in my estimation ... the NWE may become the leading 8-16 day model at this rate.

Edited by Roger J Smith
Link to comment
Share on other sites

Posted
  • Location: st albans
  • Location: st albans

as if to prove that stats can often mean very little of relevance, the best forecast chart re correlation (SFT), does not remotely show the most significant sypnotic feature for the uk which is a 1034mb high sat just east of us.

Link to comment
Share on other sites

Posted
  • Location: just south of Doncaster, Sth Yorks
  • Location: just south of Doncaster, Sth Yorks

Its been an interesting exercise but I tend to agree with the post above. What really matters at the end of a forecast is how much it actually relates to the actual chart. Correlations and whatever else, statistics are all very well but at the end of it all I would think that a look at each chart presented and a comparison with what the actual shows will be the best mark of success or failure if that is what is being tested?

Having tested various models over the past 6 or 7 years, and done checks on how one compared to the other it ALWAYS shows that the nearer you get to the T+00 time then the less variation there is between different models and each model with itself.

Perhaps an idea for another check might be, show the surface chart, and have a number of places dotted around the UK and predict at that time the wind direction, speed, temperature and any precipitation?

Edited by johnholmes
Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Part of that problem with the SFT map may be explained by my charitable gesture of filling in the missing parts of the map with estimated pressures from continuity, and looking at what I entered and then the map, possibly I went a bit high on the pressures on some of the missing points on that map. Anyway, I was planning to report on pressure differences in the final discussion, and that tends to show a somewhat different order of finish.

Link to comment
Share on other sites

Posted
  • Location: Rochester, Kent
  • Location: Rochester, Kent

Part of that problem with the SFT map may be explained by my charitable gesture of filling in the missing parts of the map with estimated pressures from continuity, and looking at what I entered and then the map, possibly I went a bit high on the pressures on some of the missing points on that map. Anyway, I was planning to report on pressure differences in the final discussion, and that tends to show a somewhat different order of finish.

Out of interest, how is the correlation computed?

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

as if to prove that stats can often mean very little of relevance, the best forecast chart re correlation (SFT), does not remotely show the most significant sypnotic feature for the uk which is a 1034mb high sat just east of us.

That's the only feature I've got close to so far!!! (down on pressure and size completely though). Last one before the actual analysis now:

post-6667-0-72433700-1300612772.jpg

post-6667-0-72433700-1300612772_thumb.jp

Link to comment
Share on other sites

Posted
  • Location: st albans
  • Location: st albans

just looking through the various entries and i'd say that ice and murcie (poss paul B) are the two efforts that stand out. quite honestly, some of the charts that have scored highly for correlation really are nothing like what has verified which makes my earlier post even more relevant. odd innit ?

as far as my chart is concerned, i'm happy to admit to it being generated purely as a day's extrapolation of the naefs T384 anomolies with the bits in between the models output (high anomoly ne scandi and wsw of the uk plus slightly low anomoly over the central med) needing to be filled in by me. the general 'feel' of the model wasnt too far off. it overstated any anomoly over ne scandi though pressure is generally above/well above average across nw europe and scandinavia. the med is showing pressure a little below what might be expected, though a little further east. what the model was not picking up was the average low heights around iceland which, given the high anomoly just to the west of the uk, could not have been expected to be present without there being a very tight pressure gradient between scotland and iceland.

to me, this exercise shows well the situation that has existed for the second half of winter whereby the energy to our north has been underestimated by the nwp in fi and slowly recognised as time has ticked down.

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

quite honestly, some of the charts that have scored highly for correlation really are nothing like what has verified which makes my earlier post even more relevant. odd innit ?

I can't do all the clever stuff with maths so it has to be pictures and crayons for me. :rolleyes:

Link to comment
Share on other sites

Posted
  • Location: Eastbourne, East Sussex (work in Mid Sussex)
  • Location: Eastbourne, East Sussex (work in Mid Sussex)

Here's the definitive Met O chart I shall use for my own comparison (probably tomorrow) - I'm not holding my breath! :lol:

post-6667-0-52725500-1300637792.png

It's been interesting and I'm still no closer to making a valid forecast/chart even a few days out!!!

post-6667-0-52725500-1300637792_thumb.pn

Link to comment
Share on other sites

Posted
  • Location: Rossland BC Canada
  • Location: Rossland BC Canada

Final report -- actual vs forecasts

________________________________________

Thanks for posting the above "actual" maps, I may eventually put one up that cannot change later (some of these links may update and change the map in which case we should make a note of that fact for anyone looking in after today).

The point about correlation was valid, and I have compared the entries both by that method and pressure difference which tends to identify the closest maps in terms of pattern. Correlation will tend to provide a good measure of wind direction accuracy more than intensity, and then we can't really assess most of our maps at all in terms of temperature because there is no upper component provided. We could probably all guess from the maps what sort of temperature regime was implied. With the exception of my map, there are no clues provided about upper level wind direction beyond what you could guess from the surface maps. (I showed some low pressure tracks -- these actually verified as to upper level winds although the low pressure areas were rather dodgy).

Before we expose our contestants to the required ritual humiliation and cleansing, let's put the major models through that same exercise. This map workshop demonstrated one thing for certain, a case of the two major models doing about as well as they ever do in forecasting from 16 days out. There might be cases with higher correlation scores than these, but not very many (the earlier 1-5 Feb study had much lower results for the major models in the first half of this time period).

Correlations are as follows, starting from day 1:

DAY...GFS...ECM

01....98....99

02....96....97

03....93....94

04....88....93

05....82....87

06....59....87

07....53....94

08....76....92

09....62....28

10....08....58

11....37

12....90

13....71

14...-20

15....58

16....64

The general comment here is that the ECM began to home in on the solution from day 8, while the GFS was still struggling a bit through day 6 (its day 12 effort was better than all but days 1, 2 and 3.)

Now, on to the really interesting part, the map entrants and their results. I have decided to list these by average pressure error rather than correlation (which I also show) because I buy into the comments from various onlookers that correlation alone cannot tell us whether a map is "good" for forecast purposes. Actually, a map with correlation 1.0 and pressure difference 10 mbs could achieve that by being 10 mbs out all over the grid, in which case it would give a reasonable forecast, or it could be too intense by a gradient increase or decrease of 10-20 mbs in which case it would give faulty wind speed and weather type implications. However, a bad map won't fluke a high correlation, if you see what I'm saying, if a map is just plain wrong from pressure differential, it won't score a high correlation.

So, the pressure difference (avg of absolute errors) and correlation scores for each map are summed up below, followed by a few comments about the effort from my subjective point of view. Feel free to add yours.

Forecast ... P diff .... corr .... comments

__________________________________________________________________

MB. ...... 5.8 ...... .75 ......suffers only from being shifted a little to the west, intensity very close to reality

PB. ...... 6.4 ...... .84 ......quite a good effort and could be used to give accurate daily forecasts

BLU ...... 9.3 ...... .25 ......stays reasonably close to reality all over the grid, lower corr due to split high

NW. ..... 10.8 ...... .80 ......our ensemble fared quite well, similar comments to MB above

STM ..... 10.9 ...... .41 ......subjectively would rate this lower than next three entries, pattern looks like 25 Mar

ICE ..... 11.7 ...... .69 ......main problem was having high too far west but otherwise quite close to reality

GP2 ..... 12.4 ...... .91 ......these two entries fared well within their limits as index-derived model maps, (contd)

GP1 ..... 12.5 ...... .77 ......and both of them gave a good indication of upper level flow and air mass distribution

TWS ..... 13.2 ......-.07 ......this map was in play with the models early on and is the actual pattern shifted north

RIC ..... 14.1 ...... .57 ......a more extreme version of what actually happened, not a bad effort

COA ..... 15.2 ......-.06 ......intensity is generally good but main features are scrambled

RJS ..... 17.1 ...... .34 ......too intense and adds a second feature that in reality is further west and weaker

STF ..... 21.2 ...... .92 ......too intense, still does well on corr with wind pattern accurate in general

LOM ..... 26.9 ......-.13 ......may correlate well with maps in 5-7 days, so from long distance, some pattern recognition

closing remarks -- Glad this is over, really. The overall results were fairly obvious about five days ago.

A few comments on maps themselves -- be sure when drawing a map that you label isobars, otherwise your contest director will have to work out your interval and write down a few on a scrap of paper. It also helps if you fill out the entire grid with isobars. There were three maps with "logic errors" in their isobars, I won't bother to embarrass their authors, but watch out for two things, isobars that are two steps apart rather than one, and isobars that poke into places they don't belong. Also if you have quirky little highs or lows, label them as such (this is a criticism that can apply to the big expensive models sometimes) -- just because there's an isobar doesn't mean the viewer can tell what isobar it is meant to be. Quite a few of the maps, on the other hand, were well drawn and easy to read.

I may think of more to say later, after reading a few other comments. But as a group, these NW forecasters should get a round of applause from the forum, just for putting together an ensemble that had such a positive verification, 10 mbs and .80 correlation is very good.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
  • UK Storm and Severe Convective Forecast

    UK Severe Convective & Storm Forecast - Issued 2024-03-29 07:13:16 Valid: 29/03/2024 0600 - 30/03/2024 0600 THUNDERSTORM WATCH - FRI 29 MARCH 2024 Click here for the full forecast

    Nick F
    Nick F
    Latest weather updates from Netweather

    Difficult travel conditions as the Easter break begins

    Low Nelson is throwing wind and rain at the UK before it impacts mainland Spain at Easter. Wild condtions in the English Channel, and more rain and lightning here on Thursday. Read the full update here

    Netweather forecasts
    Netweather forecasts
    Latest weather updates from Netweather

    UK Storm and Severe Convective Forecast

    UK Severe Convective & Storm Forecast - Issued 2024-03-28 09:16:06 Valid: 28/03/2024 0800 - 29/03/2024 0600 SEVERE THUNDERSTORM WATCH - THURS 28 MARCH 2024 Click here for the full forecast

    Nick F
    Nick F
    Latest weather updates from Netweather
×
×
  • Create New...