Better Than The Models ?

[bluearmy]

sorry, but i hope i'm not the only one who isnt very impressed with today's 'match up'.

i think pinning the charts down to specific days from several months out is a bit ridiculous. surely, when we have got to the 10th feb, we should look and see if any of nick's forecast charts bear more than just a general resemblance here or there to any actual chart between now and then. of course if it does, its likely to be one which shows a deep depression which will no doubt impress many on here. has anyone done the stats to show how often we get a deep depression in the north sea between 25th jan and 12th feb ???

i'm not being over critical. infact i think by asking to ignore the timing issue, i'm being more than fair. to say todays match up is good because there is a westerly flow into nw scandi and pressure is high in ireland isnt impressive for me. nick shows a centre of high pressure circa 1040mb over ireland whereas we have a ridge circa 1025mb. nick shows another centre of high pressure over spain whereas we have a trough. i think we should be cautious until the period is ended and then try and find some match ups. obviously, nick's thoughts on the pattern that would drive these storms are not correct but if a major storm in the north sea at this timescale in winter is uncommon and we get one then thats a result of sorts i think. (though i fear it may not be so rare)

and maybe next time, a bit less trumpet blowing and certainty of extremes will help the forecast ??

[nking]

sorry, but i hope i'm not the only one who isnt very impressed with today's 'match up'.

I hope you are in the minority that look forward to reviewing the data as and when it was forecasted to occur. This is a similar approach for "model output discussions", they don't wait 10 days before they comment on models. However, your suggestion can be achieved by not visiting this topic until the 10th Feb (only joking).

[AtlanticFlamethrower]

sorry, but i hope i'm not the only one who isnt very impressed with today's 'match up'.

and maybe next time, a bit less trumpet blowing and certainty of extremes will help the forecast ??

No... next time we'll have a control with one of MB's critics, Pieman, you, Steve Murr, independently come up with their own specific forecast. We'll see who's closest.

If it is all down to random chance, as you seem to think, you are all just as likely to get it right as MB. Or even more right.

Ed: MB will pick the date and will ask us for our forecasts before he posts his. I may even have a go myself

Edited January 28, 2011 by AtlanticFlamethrower

[Weather-history]

I'm still of the opinion that if the extreme event doesn't come off then the forecast is somewhat a failure. To me, the use of this technique is too detailed at this stage, its like trying to build Rome in a day. Put it this way, man didn't develop rocketry and then tested the first launch by trying to put himself on the moon with it.

Edited January 28, 2011 by Mr_Data

[Harsh Climate]

I would like to see his methods and how he came to his forecast, that way we would be able to see if his forecast was thoroughly made using some sort of testable technique. Because at the moment it all just looks likes sensationalist guess work??

You kind of get the feeling that ''any sort of deep depression'' that may develop in or outside his forecast timeframe, he's gunna try and put some sort of spin on it and try and pass his forecast as 'nearly right', but that he made some 'school boy errors (as he says) that caused his forecast to be wrong.. When lets face it, soon we will have a positive NAO aswell as a positive AO....... hardly suprsising our weathers about to turn more unsettled with the chance of numerous depressions tracking west to east?

http://www.cpc.ncep.noaa.gov/products/precip/CWlink/pna/nao.sprd2.gif

http://www.cpc.ncep.noaa.gov/products/precip/CWlink/daily_ao_index/ao.fcst.gif

The bit that gets me, is if MB's forecast goes badly wrong when he claimed to be 'nearly 100% certain a storm of the century would hit' 'I cant see this not happening', how can he claim to have made school boy errors, missed pieces of information when even he doesnt have conclusive evidence that his methods work?? It comes across as arrogant that no matter what happens, we are to still believe that his methods work not even thinking that the whole idea might just be wrong/impossible..

[peterf]

i'm on the fence eitherway.but solely for some of the responses posted on here,i hope he's bang on the money regards his prediction.

that would make my day i tell ya.

[La Bise]

Harsh Climate is hereby authorised to talk on my behalf on this thread, splendid post Sir!

I have my own little prediction at play here, so far I'm quite confident it will come to pass (see previous posts for what it is)

[Harsh Climate]

Harsh Climate is hereby authorised to talk on my behalf on this thread, splendid post Sir!

I have my own little prediction at play here, so far I'm quite confident it will come to pass (see previous posts for what it is)

lol, thanks Im just saying it as I see it.

I think at some point if MB's methods are to be even considered taking seriously we need:

• A high percentage of his forecasts being accurate.
• To see testable 'methods' and 'theory's' that can be scrutanised.

Like I said the other day Its like a battle of statistics v mathmatics, with MB being the statician and the more current methods of weather forecasting as the mathmatics. The problem is I see statistics as only a small piece of the jigsaw rather than the person using mathmatics and equations to put the jigsaw together.. So basically you cant work out A+B+C when you only have 'B'...

[Roger J Smith]

Continuing the validation for 3 Feb as we count down to day 6 now.

The two main models are "locking in" now showing higher correlations to their previous efforts and to each other. Both GFS and ECM are now showing a broad westerly flow of moderate intensity to the north of high pressure ridged east-west near the southern boundary of the validation grid (45 N). The GFS is developing a distinct low west of Ireland that is progged to track across Scotland night of 3-4 Feb and into Denmark. This does have a fairly strong wind field by 06z 4th although in the 40-60 mph range rather than extreme values. The ECM is less robust in its development through the period.

More specifically, the GFS correlates at 0.9 with its previous effort (day 7). The correlation with RJS (which was a 22-day toss at the dartboard) continues rather high at 0.52 (this value is slipping from a peak of 0.6 at day 8.). The correlation with MB is 0.08. To give some idea how the GFS is performing over five days, the correlation from day 10 to day 6 is 0.82. What we can say at this point is that the "RJS" has shown a tendency to significant correlation with the GFS during this first half of the validation period, whether both are moving towards an actual solution remains to be seen.

Meanwhile, the ECM correlates at 0.83 with its previous effort (day 7). It must sit about midway between our two experimental forecasts because it correlates at 0.36 with RJS and at 0.32 with MB. The two main models (GFS/ECM) are correlated with each other at .94, a statistic which continues to increase (as would be expected). The main difference between the models is the short wave west of Ireland, a feature which is now only 5 deg west of the RJS model on the GFS but which does not become a separate entity until 4 Feb in the Baltic on the ECM. As a measure of how much trouble the ECM is having with its longer-range forecasts, the correlation from day 10 to day 6 is a very low -0.54. In other words the ECM has undergone a significant pattern change in its forecasts for 3 February.

Meanwhile, intense storm development that had been featured on previous model runs in the zone between south coast Iceland and south-central Norway has weakened somewhat and shifted more towards the Norwegian Sea. Although the later part of the validation period and beyond show moderate to borderline strong winds at times, they don't appear as extreme as on last night's or yesterday's model runs.

It is interesting that today is the first day in this exercise when all correlations were positive.

The really interesting part of this exercise will be the final correlations with reality on 3 February, and a check back on the 10 days of forecast maps for the main models, to find out where in that progression the two experimental models do a similar job.

I had a quick look at the 144h UKMO and GEM, these appear broadly similar to the ECM and would probably not correlate much differently with the two experimental models than what we see above.

Edited January 28, 2011 by Roger J Smith

[bluearmy]

ooh dear - all i said was that i wasn't impressed with todays match up and apparently i'm a 'critic' or is that heretic!! i also said we should give nick a 2 week window for his forecast charts to verify rather than pinning it down to specific days. the 'control' is surely the statistical likelihood of a winter storm in this period of the winter. if his forecast doesnt verify and no one really believes we will see winds of 160mph, then should we get a significant winter storm in the north sea, i'd like to know the odds of it occurring anyway. that was my main point. as for the trumpet blowing - i find that a bit of magnanimity goes a long way.

[La Bise]

I find your statistical excersise very interesting RJS, it's particularly good to see how models evolve their scenarios and the correlation between different runs and indeed models.

[MurcieBoy]

Not having a go, but we all know how "fluid" the models are more than a few days out. The deep lows close to the UK appearing on the models have been getting closer to the target day of 3 Feb 2011 on each run. I believe this is the closest, just 3.5 days after the "target day".

I appreciate GFS likes to do this, but its getting closer folks, just one day away now from the target day:

[carnoustie]

Whatever happens, I think that MB is to be congratulated on his forecast.

The forthcoming storminess was forecast 2 months ago and judging by current output it's gonna be pretty close.

Fair enough he maybe overcooked the severity (GFS does that everyday almost) but the general synoptics are close enough.

Some people on here need to grow a pair of balls and give credit where credit is due.

Read some of the so-called "expert" analysis on here (and TWO) if you want a dog biscuits forecast.

Dog biscuit......wtf. :p

Edited January 29, 2011 by carnoustie

[PB22]

Whatever happens, I think that MB is to be congratulated on his forecast.

The forthcoming storminess was forecast 2 months ago and judging by current output it's gonna be pretty close.

Fair enough he maybe overcooked the severity (GFS does that everyday almost) but the general synoptics are close enough.

Some people on here need to grow a pair of balls and give credit where credit is due.

Read some of the so-called "expert" analysis on here (and TWO) if you want a dog biscuits forecast.

Dog biscuit......wtf. :p

My first (and probably my only) post on this thread.

Credit could, and can be given to some extent, if Nick was forecasting a general pattern. However, Nick has given a very specific forecast with specific pressure readings for specific locations. Now, no disrepect to Nick here, but anyone who knows a thing or two about meterology will acknowledge the fact that it is impossible to come up with detailed pressure charts two months or so before a specific day like Nick has and expect to be correct. Whilst the general pattern may be right, Nick's forecast will have to go down as being wrong purely because he is being so specific and he has indicated the importance of this accuracy himself. We can already see that the chart Nick came up with for yesterday was technically wrong, although the overall pattern wasn't too far out. A 1005mb low over Spain where a 1030mb high actually was forecast to be can not go down as being right. At the same time, a 1025mb ridge of high pressure over Ireland is not the same as a 1040mb high sitting there.

I don't begrudge anyone trying to come up with new methods of long range forecasting, but I do feel very strongly that it is a case of trying to run before you can even walk. Drawing pressure maps for 2 months out with self proclaimed high confidence is never going to work, whatever the methodlogy. However, going on and drawing a sequence of pressure maps for a specific time period 2 months out is just a waste of time to be frank, let alone going on to predict an event that is actually probably scientifically impossible to actually occur! No disrespect to you Nick, but it will never work. Stick to general patterns and give yourself some leeway and you may be onto something, but don't get into specifics because you will be fighting a losing battle from day 1.

[Alan Robinson]

i'd like to know the odds of it occurring anyway.

Odds, meaning the probability of it happening divided by the probability of it not happening? I'd personally just like to know the probability of it happening, that makes it easier for me to understand.

[Alan Robinson]

I find your statistical excersise very interesting RJS, it's particularly good to see how models evolve their scenarios and the correlation between different runs and indeed models.

I am personally skeptical about statistical methods, because probability applied this way often means forecasting the future assuming it will be more or less the same as the past, ignoring that in the meantime, circumstances change.

Take the ability of ships to survive damage for example. For many decades we designed passenger vessels so they had a certain number of watertight subdivisions, and said that they must be able to survive flooding of one, two or three of those compartments, depending on the numbers of passengers and other criteria. The bright-sparks at the International Maritime Organisation then decided this wasn't good enough, and recently introduced a probablistic approach to the same safety issues. Now, new ships can have all kinds of higgledy-piggledy compartments - allowing cruise ship designers even more freedom to install bowling alleys and snooker tables - and safety is decided by how likely a vessel is to sustain damage, and then how likely it is the vessel can survive that damage. The probabilities for this assessment are based on - guess what - ships in years gone by where numbers of vessels and their routes differed, weather differed, and to crown it all, they had the number of previously decreed watertight compartments. It is therefore good to see probablistic weather models evolve.

Disregarding things such as dice and card games and pulling coloured balls out of bags randomly in various combinations or sequences, probability assumes to a great extent that the future will be like the past. Presumably this explains why predictions can go very wrong, even though it looked fairly certain before the event.

[Weather-history]

I stumbled across this in my archives and its a list of low pressure readings, written in 1868, that was recorded at Galshiels on the 5th of February

1857: 965mb

1858: 972mb

1859: 958mb

1860: 972mb

1861: 955mb

1862: 982mb

1863: 958mb

1864:-

1865: 965mb

1866: 958mb

1867: 942mb

1868: 975mb

I have seen reports questioning whether the 25th October is a date where gales are common

Edited January 29, 2011 by Mr_Data

[Roger J Smith]

I would say looking at the 00z output that the GFS is moving closer to a high correlation with MB, while the ECM is about steady-state with previous 12z data. Don't have time to add these other hours to the exercise and will post later today when 12z is available.

Some of the comments above seem totally misinformed as to what is going on here. A correlation study has nothing to do with probability, it's a test of significance. I think the poster is confusing the validation study with the forecasting method. Also Paul B's remarks, while reasonable in terms of one step at a time, actually seem to ignore the fact that with very specific predictions like this, it is easier to assess the exact scientific validity of the proposed new method. Of course at some point it would be necessary to give a clear statement of the new method, otherwise all we know is that somebody can do something with .3 to .7 correlation results expected from a number of trials -- what if that person then loses interest or passes away? Science gains nothing.

But I don't see it as a weakness to produce specific maps, I've done this for many years mostly in-house behind the scenes to test out how close the theory is coming to matching reality. People who think this is all random should carefully check the post I've made on Americanwx.com where I track events on timing line one in January. There is a rather obvious correlation between low pressure on this timing line and a set of events in an agenda. What might be the cause and effect here is subject to debate, but as with gravitation, we would ignore this at our peril whether we know the cause and effect or just realize the relationship exists. I've found timing lines three and four which run through our map grid for this thread, have more blocking and that low pressure tends to be shoved to the extremes northwest and southeast along the timing lines when this happens. This has been the case for a while now since the mild spell ended, but as fortune would have it, the blocking is now breaking down so there should be stronger lows moving east closer to the UK and Ireland. Energy peaks are timed for 31 Jan, 3 Feb and 6-7 Feb and I would expect a pressure graph for any selected station to show these three waves, with a lag time depending on position relative to timing lines. Timing line three runs NW-SE through Valentia, timing line four runs NW-SE through northern Scandinavia into western Russia. You may have noticed in the past that there's a tendency for low pressure to exist on these timing lines at similar times, but when the agenda gets crowded, you can have a chain of low pressure linked up in a wave train.

MB hasn't said much about cause and effect, but in my case, a strong event on 3 Feb would be timed from the new moon which is at 03z. Location would depend on instantaneous location of timing lines three and four, these have been lagging west of equilibrium in recent days from observed events. In my 22d forecast I allowed for a slight westward lag but I may have underestimated it slightly. Note also from GFS global output, a very strong low will be approaching timing line 2 at this event time (that's the one coming out of the northeast U.S. on 2 Feb and into eastern Canada on 3 Feb).

I would say that if MB achieves correlations in the 0.3 to 0.7 range his next step should be to give a concise, no-mumbo-jumbo statement of method, and I would certainly like to see forecasts for energy peaks around 3-5 March, 19-21 March and 2-4 April that I could compare to theory I'm developing and then the model runs coming into those periods. If there's overlap of method, then we could also look at a correlation of "blended" maps (average of the two pressure grids). I will do that statistic on today's 12z output to see if blended forecast is out-performing either individual map. Looking at the 00z GFS I can see that blended pressure grid would have quite a good fit, as both our tracking lows (in two different sectors) are on the GFS output.

[Roger J Smith]

Meanwhile, to address this question of randomness, as in what are the chances of "something like this" happening at random, that all depends on how strictly you define "this" and the time period.

If you were to say, what are the chances of wind gusts over 100 mph on any date 1-5 Feb in the North Sea region, that would probably be about 1 to 2 per cent at random.

If you dropped the criteria to wind gusts over 70 mph on any date 1-5 Feb in the British Isles, North Sea, or adjacent western Europe, that might rise to 3-5 per cent.

If you were even looser with definitions and asked what are the chances of wind gusts over 60 mph in that grid any time from 27 Jan to 10 Feb, I suspect the chances would rise to at least 50%.

This illustrates why I am doing this rather precise validation study. I wanted to head off any pointless discussions if some strong wind event struck just outside the validation period or there was a marginal wind event in the validation period, both fairly high probability outcomes. This is after all the northern hemisphere in winter, but as we know, long spells of rather quiet weather come and go, so the more active periods seem unusual. On a statistical basis, we have to know what we're defining to get these percentage chances.

People in the western Isles, the Shetlands and places like Belmullet or coastal Donegal shrug off 60-70 mph gusts as part of the local climate, so I would say gusts of 80 mph or more form the thresh-hold for "very strong winds" in this region. But a wind gust of 60 mph in London would be more remarkable. Anyway, it's all being assessed by the numbers, after we more or less agreed to cut MB a break on the more extreme numbers in the video.

The most significant indication of a "hit" with the experimental forecast would be a correlation over 0.5 for either map time or some time within 24 hours of map time. Anything you could add to that would be anecdotal, for me, the pattern correlation is the main question we need to address. When you have that, you can do the second-order research about which cases will develop extreme winds.

[AtlanticFlamethrower]

Here's the GFS 00z

I would say that if MB achieves correlations in the 0.3 to 0.7 range his next step should be to give a concise, no-mumbo-jumbo statement of method, and I would certainly like to see forecasts for energy peaks around 3-5 March, 19-21 March and 2-4 April that I could compare to theory I'm developing and then the model runs coming into those periods.

If MB accepts this challenge we should all - especially including those who know least about weather forecasting - have a go at drawing a map/give a precise synoptic forecast for this date.

[OldMetMan]

Some further thoughts on the latest model output in relation to MB's forecast.

In broad terms, the models indicate an increasing zonal flow from now until the first week in February, but with the possibility of one or more quite intense lows moving to the N of Scotland and heading into Scandinavia. High pressure is generally shown to be retreating steadily to the S and SW, except the NOGAPS which brings a strengthening HP in mid-Atlantic by 4th February.

The basis of what MB has predicted relies on having fairly intense HP just to the W of the UK with low pressure coming from the NW and, as far as the models are concerned at this stage, that doesn't seem very likely, at least not in the time frame of up to 5th February.

However, there are still features of the current synoptic situation that bother me, which I have referred to before. Specifically, the very southerly jet over N Africa, with associated low pressure over the Mediterranean. Whilst this remains, we still effectively have a block, albeit at a more southerly latitude than previously. The models do show the upper high of this block now over us moving to the E and gradually collapsing but we will still have that upper trough over the Med. Now, if we had the very common situation of HP from the Azores E to the Med, or just across S Europe, then I would see no complication arising from a strong SW jet, as forecast currently, taking control of our weather.

But, based on how I have seen developments take place before in the past, whilst that southerly block exists, there remains the possibility of the scenario I described in my last post, namely a distortion in the upper flow - trough over E Canada/US, ridge in mid-Atlantic, corresponding trough over or to the E of the UK. Long trains over low pressure originating out of Canada/US frequently end with a major low pressure developing at the end of the train, sufficient to deepen the upper trough in that area - i.e. over E Canada and setting up the upper long wave pattern downstream as I have just described.

As I said, the only model that hints at this presently is the NOGAPS. If such a development did happen, I am not sure it would occur within the time frame in question here but I do believe it's a possibility.

As before, if any more technically knowledgeable posters wish to explain why the above couldn't happen, please feel free to weigh in!

[Coast]

Anyway, it's all being assessed by the numbers, after we more or less agreed to cut MB a break on the more extreme numbers in the video.

But I don’t think many of us can ignore the wind speeds and the associated predictions of flooding etc Roger.

The headline of the forecast is ‘The Great North Sea Storm’ and by Nick’s choice, he has given some specific details (with high confidence) of when and where these events will occur. If I look up ‘City trader predicts areas of atmospheric pressure’ on Google I don’t get his forecast. If I look up ‘Great North Sea Storm’ (or even just North Sea Storm) I get his video and the associated discussion threads fairly high in the rankings. If Nick said he was going to do 185mph up an Autobahn in his Lambroghini next week, I would want to prove he has, not just worry about the fuel consumption and how far is foot was off the floor at the time! :lol:

Nick could have picked any other 5 day period this year to predict and comment on but I’m sure he’ll agree, he has seen a massive storm coming (one of the top 5 in the last 300 years) and has used his method to forecast not just pressures and the areas they will be in, but what he feels will be the result from those weather charts – in quite graphic and precise detail.

Please don’t feel I’m pre-empting my decision on this particular forecast just yet, I am merely stating that the specific windspeeds, flooding etc should be considered in the overall appraisal and I’m sure that many people including Nick himself will want to use this as part of the judgment on the success or otherwise of the prediction.

[La Bise]

A couple of points:

- MB gave a very precise forecast and indicated a very high degree of confidence. Had it been a more general pattern forecast, I doubt there would have been much criticism. Live by the sword, die by the sword or something along those lines...

- No explanation other than something straight out of a New Age pamphlet about the method. Contrary to what some seem to assume, I'm pretty open minded about new ideas but I do resent being thrown a few vapid explanations that seem to be completely at odds with everything we know about our atmosphere and indeed life in general. I did not, nor did any of the sceptics come up with those statements that seem to be completely glossed over in favour of an "everything is possible" approach. I've highlighted them in a previous post.

No one does forecast as precise and as far in time as MB has done using conventionnal methods, demanding of the sceptics to put up or shut up in some kind of competition is neither here nor there. It would be a bit like asking people doubting someone saying he can fly to jump themselves out of the building and do better...

I've had an interest in many things people, some that would even have some of MB's biggest supporters wince in disbelief, and one recurring problem that crops up when you investigate an anomalistic subject is that of "knowledge by arcane", someone claims to holds an essential truth that would give an explanation for the anomaly studied and much more beside but refuses to divulge anything but tantalising and/or bewildering snippets. That leaves the investigator in a quandary as nothing can be tested rigorously. The investigation, even if some or the entirirty of the anomaly is hard to explain in more conventionnal terms, generally dies as we are then in a situation which you could call as "revelation truth" and we enter another realm together, that of belief and faith statements.

[MurcieBoy]

As always, RJS and OldMetMan thanks very much for your posts today. RJS you hit the nail on the head re why one would wish to produce specific detailed charts weeks/months/years ahead. Its not arrogance, I just want to see the deviations with the actual (as I have explained many times, the deviations will help me refine/fine tune the method). It is worth also repeating: I am not expecting the Storm forecast charts to verify 100% (as stated in video), but equally I would be disappointed if they were widely out (and I couldn't fathom out why).

OK, here are the comparisons for the first five days of Feb:

1 Feb

2 Feb

3 Feb

4 Feb

5 Feb

[pottyprof]

- No explanation other than something straight out of a New Age pamphlet about the method.

To be fair, Nick has said the method is not up for discussion for now so I don't think any further comment is needed... Unless you want to sound like a broken record?

No one does forecast as precise and as far in time as MB has done using conventionnal methods, demanding of the sceptics to put up or shut up in some kind of competition is neither here nor there. It would be a bit like asking people doubting someone saying he can fly to jump themselves out of the building and do better...

Again, to be fair, Nick is not asking you to accept anything. He has said it is an experimental forecast and you can judge its performance after the dates he has set. So yeah I suppose it's a put up and shut up as you are getting no more and no less than Nick told us all at the beginning.

I've had an interest in many things people, some that would even have some of MB's biggest supporters wince in disbelief, and one recurring problem that crops up when you investigate an anomalistic subject is that of "knowledge by arcane", someone claims to holds an essential truth that would give an explanation for the anomaly studied and much more beside but refuses to divulge anything but tantalising and/or bewildering snippets. That leaves the investigator in a quandary as nothing can be tested rigorously. The investigation, even if some or the entirirty of the anomaly is hard to explain in more conventionnal terms, generally dies as we are then in a situation which you could call as "revelation truth" and we enter another realm together, that of belief and faith statements.

Nick has said he isn't divulging details of his method until he has the proof that his method works. I think from that, he doubts the accuracy. Nick has stood by what he has said and from that you may assume that, for now, you will get no further information.