Better Than The Models ?

[TEITS]

I sincerely hope MB answers my question. For example the 06Z GFS F.I charts are in stark contrast to last nights 18Z as HP around Greenland/Iceland is now replaced by low heights. So all I wish to know is how does MB method cope with the chaotic nature of our climate.

[La Bise]

You're going to wait for a long time TEITS...

[Coast]

In a week of Storms over the UK

Ermmm, that'll be two days of windy gales in Scotland I think

As for Sally Bercow

[dogs32]

^^ Meso, I'd like to see Spurs play Barcelona at some point (after the latter have despatched the Gooners....)

Right here is the last comparison with GFS and the Actuals:

5 Feb

Actuals:

28 Jan

30 Jan

1 Feb

2 Feb

3 Feb

4 Feb

Murcie Boy whats your prediction for Tornados this year... what month is the worse..any thoughts

Edited February 5, 2011 by dogs32

[Rob]

Please be correct on that prediction!

Fingers crossed!

[MurcieBoy]

Re Tornadoes: I will quietly reflect on the last three predictions in the next few weeks, do some research and will hopefully be able to come back with some video predictions.

The "method" clearly needs refining (I knew this from the start, before I published any predictions) and I needed "feedback" from Nature to tell me what refinements I needed to make; that was the whole purpose of publishing specific and detailed Experimental Forecasts, which have now provided me with the clear variance information that I needed and sought at the outset. This variance information will be the centre of my follow up research. Unfortunately, I cannot answer certain recently raised questions from posters, as at this stage I have a host of healthy key questions for myself and it is up to me to try and fathom out the answers (this is the part I enjoy the most).

Glad to see a few Spurs fans on here!

Edited February 5, 2011 by MurcieBoy

[Rob]

Glad to see a few Spurs fans on here!

Definately is, going to be a good game today!!

[Roger J Smith]

On the secondary validation (see post 190), MB scored reasonably well in the criteria for intensity of upper and surface winds, which gave a five-day validation window (1-5 Feb) and scored only from the highest upper and surface winds recorded in that period. His total score was 34/100 on this validation. I had similar scores on all but these two variables and scored 25/100. However, if the intensity had been scored daily like the other variables, the scores would have been more similar. The upper level winds were most intense around the 4th 00z during the period.

Some people have developed a faction here which basically believes in the premise that weather patterns are random beyond a certain time period and that nobody can (or presumably should) search for patterns or predictive systems.

I'm not sure how they justify the expense that (through their taxes) national meteorological agencies put forth in a search for exactly that objective, perhaps the belief should be restated as "nobody except the designated authorities, the scientific community as we define it, and anyone who is already earning a large salary and supposed to be doing this can or should look for patterns." This would perhaps be more accurate, and after all, isn't it greater accuracy that a weather forum should be all about?

I'll just leave this final thought -- "chaos theory" is more or less a meaningless concept, it is only applicable to what we don't know. Before scientific discovery takes place in any given field, chaos theory is what prevails. For the New Guinea tribesman, presumably atomic theory is chaos theory. It is more intellectually honest to say that this is simply a scientific question that remains unanswered, for which further work is required, than to throw up your hands at the complexity of the challenge and say the explanation is chaos theory. Chaos is just a mass of data that we don't understand. We read into that lack of understanding that there is no paradigm by which it could be understood, that we are just stuck with the data coming at us randomly (except for the next five days) and the implication that must therefore be taken is that we are already at the peak, the summit of our knowledge and understanding, because otherwise, instead of chaos we might perceive unfinished work instead.

Anyway, the history of science is probably littered with such illustrations of nay-sayers who thought that science would go no further than what they themselves could imagine. Fortunately, the history of Britain is somewhat less shabby.

[Coast]

Sorry, a bit late with this comparison (good god, I have a life outside of NW!!! :winky:) and as I posted earlier, its only basic but I will do an overlay Monday just for the record:

A brief analysis and you can see that the low off Norway yesterday still corresponds, but although I'll admit its windy out today, the Isobars are in a completely different orientation and the wind flow/direction too. A quick check of the links I posted a couple of days ago reveals no current storm surge in the North Sea and nothing in terms of flooding in the forecast area.

I need to sit down (probably with a glass or two of rum) and collect some thoughts together, but as we near the end of the forecast period, I can imagine that the wind is going to be with us for a couple of days yet (so therefore outside of the period under scrutiny) and seems set to continue in a W/SW direction driven by the Atlantic low.

[La Bise]

Before:

I could be proved wrong, but I cannot see how this forecast will not come to pass. If this forecast is not closely fulfilled, I shall spend an awful long time probably trying to work out where it all went wrong!

If my level of expectation of the 12 Nov ’10 prediction was say 75% positive; my expectation for this forecast is considerably higher.

After:

The "method" clearly needs refining (I knew this from the start, before I published any predictions) and I needed "feedback" from Nature to tell me what refinements I needed to make; that was the whole purpose of publishing specific and detailed Experimental Forecasts, which have now provided me with the clear variance information that I needed and sought at the outset.

RJS, you totally misrepresent what chaos theory is, it is not the void before a discovery is made, chaos is very much the essence of life, the universe and everything (thank you Douglas Adams...), people working on chaotic systems, be it the weather or our very own body, do try to find patterns but, ultimately, not everything can be explained nor predicted (itself a rather repugnant notion if you think about it...)

Or rather it can but then we enter the realm of religion and mysticism....

A highly recommend read would be this:

http://www.amazon.co.uk/Impossibility-Limits-Science/dp/0099772116

Chaos theory is one of the great advances of science, before it, there was very much a sense that a very strict cause to effect paradygm existed (Newton famously said that if he knew the place of every particle in the universe, he would be able to predict the future), equally amongst religious leaders as scientists, now there is a far more fluid acceptance of uncertainty and indeed limits.

Chaos theory ulimately was a step forward in freeing us of the dictatorship of God, of revelaed truth, of us being enslaved in some kind of deterministic universe, ruled by recurring patterns as if we were automatons without any concept of free-will.

[Coast]

An interim thought as we reach the end of the forecast period.

Nick's forecast was very specific and contained detailed information on events, meteorology, locations and timing. Nobody but Nick himself has undertaken, or has even asked, to be so precise about this prediction.

Because the forecast is so precise, should we be critiquing it with the same precision or are there areas of the forecast we are able to forgive or forget, because a 'certain percentage' or 'in general' our perception of some of the things was that it was close, spot on, or indeed completely wrong? I suspect he would be disappointed if we didn't consider the big events that were highlighted in the video as well as the general patterns - it's what he thought would happen with a great deal of confidence after all.

I am going to take all of the stated predictions, including the headline events and the detailed weather maps in making up my mind on this forecast that used Nick's 'DNA' method.

[AtlanticFlamethrower]

Chaos theory ulimately was a step forward in freeing us of the dictatorship of God, of revelaed truth, of us being enslaved in some kind of deterministic universe, ruled by recurring patterns as if we were automatons without any concept of free-will.

So chaos theory is an essential part of your religious value system? Those appear to be moral, rather than scientific arguments against finding patterns out of chaos.

I know this is off topic; it's the end of the thread!

[MurcieBoy]

What a day in the Premier League! Nice comeback by Newcastle; great to see Spurs win!

Back on topic: I am sure the Storm Forecast will polarise opinion; these two divergent posts on another forum sum it up (made me chuckle):

[TEITS]

Back on topic: I am sure the Storm Forecast will polarise opinion; these two divergent posts on another forum sum it up (made me chuckle):

Well my view is the forecast was unsuccessful not just synoptically but weatherwise. Let me give you an example why I say this. Recently we had a very shortlived E,ly flow that gave us max temps of 2C with min temps dropping to around -4C. Now prior to this I did predict an E,ly. However can I claim success for my prediction? Answer is no because although the flow was from the E, synoptically and weatherwise it wasn't what I was predicting.

If your forecast wasn't as detailed and you just said "around this period I expect deep LP and gales" then I would say yes your forecast was successful. However as your forecast was so detailed and synoptically wrong then you cannot claim success. This is what im trying to say to you if you produce such detailed forecasts at long range then to be honest you face a losing battle. Simple reason is because its impossible and you still haven't answered my question about the chaos theory and how you factor this into your forecasts.

Take my advice you will recieve more credit for admitting you got the forecast wrong rather than suggesting it was right!

[La Bise]

So chaos theory is an essential part of your religious value system? Those appear to be moral, rather than scientific arguments against finding patterns out of chaos.

I know this is off topic; it's the end of the thread!

Erm...I'm not arguing against looking for patterns in a chaotic system, that's what most scientists are striving to do, I'm arguing against the idea that everything fit in a nice little straighjacket with an explanation for everything if only you look hard enough. This presupposes a kind of deterministic order that essentially turns us into pawns in some kind of game. Which is where the emergence of chaos theory has been an important step into moving science away from any religious/superstitious hangover.

Naff all to do with religious belief systems...

[Weather-history]

I agree with TETIS.

Take this analogy, say I develop a dart gun that I claim can fire a dart and hit the bullseye of a dartboard 200ms away. I fire it and it instead hits outside the scoring rings. Is my claim a success? No because I said I would hit the bullseye and it didn't

Now take the same analogy but this time I say that I can hit a dartboard with this dart gun but not specify a target on the board. I hit the dartboard also outside the scoring rings. Is my claim a success? Yes because I said I would hit the board, it was irrelevant what I hit on the board, so long as I hit the board.

The difference in success and failure here is all because I was specific on one and general on the other.

Edited February 5, 2011 by Mr_Data

[La Bise]

Darboard analogy, football analogy...Inventive crowd on here

[TEITS]

I agree with TETIS.

Take this analogy, say I develop a dart gun that I claim can fire a dart and hit the bullseye of a dartboard 200ms away. I fire it and it instead hits outside the scoring rings. Is my claim a success? No because I said I would hit the bullseye and it didn't

Now take the same analogy but this time I say that I can hit a dartboard with this dart gun but not specify a target on the board. I hit the dartboard also outside the scoring rings. Is my claim a success? Yes because I said I would hit the board, it was irrelevant what I hit on the board, so long as I hit the board.

The difference in success and failure here is all because I was specific on one and general on the other.

That analogy perfectly sums up my feelings on this forecast.

[VillagePlank]

One of the key components of chaos theory is sensitive dependence to initial conditions, of which I've blogged about, here.

This does not preclude prediction of the future, however. The clearest example is the use of a chaotic system to determine to the time - the pendulum. Although the system precisely (ish) oscillates and it does so because it is bound by an attractor, it also has properties such as it's final resting place that are completely dependent on it's initial starting conditions. And very small changes affect the outcome greatly. It's helpful to remember that chaos helps us to extract the inherent structure in apparently random systems (which, of course, are not random)

So, in principle, what has been attempted here is possible. Without knowing the "method" however, it is impossible to determine whether or not these factors come into play with this forecast. But what we do know is that dynamical weather models are sensitive to initial conditions.

I'll just leave this final thought -- "chaos theory" is more or less a meaningless concept, it is only applicable to what we don't know. Before scientific discovery takes place in any given field, chaos theory is what prevails. For the New Guinea tribesman, presumably atomic theory is chaos theory. It is more intellectually honest to say that this is simply a scientific question that remains unanswered, for which further work is required, than to throw up your hands at the complexity of the challenge and say the explanation is chaos theory. Chaos is just a mass of data that we don't understand. We read into that lack of understanding that there is no paradigm by which it could be understood, that we are just stuck with the data coming at us randomly (except for the next five days) and the implication that must therefore be taken is that we are already at the peak, the summit of our knowledge and understanding, because otherwise, instead of chaos we might perceive unfinished work instead.

I'm sorry Roger, but that is simply plain incorrect.

Chaos theory, even though the term is overused, and cliched, now, is a specific property of dynamical systems of which weather, and climate are a part, and of which is mathematically describable. Indeed, it was a weatherman, who 'discovered' it. Actually, the idea had been around long before Lorenz described the effect quantitatively.

James Maxwell knew of it's existence (but did not have computers to model it) way back in 1877 -" There is a maxim which if often quoted, that 'The same causes will always produce the same effects' There is another maxim, which asserts 'That like causes produce like effects.' In a great many physical phenomena this condition is satisfied; but there are other cases in which a small initial variation may produce a very great change in the final state of the system'"

Chaos theory has produced many great pieces of work, including Poincares three-body proof. A particularly relevant hero of mine, Mandelbrot, who recently died, was anathema to the mathematical community at the height of his work at IBM since what he suggested went against all the group-think that there was at the time. If a student of his hadn't become the editor of a rather obscure journal we would never have known about the word - 'fractal' and we certainly wouldn't know that the coastline of Britain was of infinite length.

Edited February 6, 2011 by VillagePlank

[MurcieBoy]

Just for the sake of completeness, here are the complete set of actuals:

Many thanks to RJS for his objective appraisals and conclusion.

Folk can subjectively debate to the n^th degree whether the Storm Forecast was a hit or a very wild miss, or whatever in between; as already stated, my focus now lies in objectively researching the reasons behind the variances.

Finally, I'd also like to say many thanks to Coast, the other moderators and everyone who positively contributed to this thread.

[Harsh Climate]

Well I've waited untill after the event before making my final conclusion and this is the way I see it.

When I first saw MB's forecast I thought 'here we go again, more sensationalist nonsense', I said if MB can get this forecast nearly right or right I'd then open my mind more and say 'maybe just maybe he may be onto something and with further accurate forecasts he must be taken more seriously..'

The reality is MB's forecast was a disaster, there was no great north sea storm, the synoptics were massively different to what he forecasted and theres not one FACT MB can bring up to support his forecasting method been accurate.

Iv'e got nothing wrong with MB coming up with new ideas its when after his failed forecast he can still try and claim it to be some sort of sucess, at the end of the day, a deepening low and sever gales towards iceland/scotland in winter is hardly uncommon!

Then we have the conflicting changes in views of MB, one minute when his forecast is looking like totally being wrong he claims to have made 'school boy errors', then when in his eye its looking good, he tries to back up his predictions with random charts saying 'thats what he expected to happen and why he has made such a forecast', then the bit where he calls his forecast 'experimental' even though he's near 100% cionfident?? It just doesn't add up and any one with a keen eye would think he was waffling on, trying to pull the wool over our eyes..

Ive found the tread a good read so I dont want my conclusion to seem too harsh, but its when youve got people like GLACIER POINT and RJS working very hard with good understanding and 'testable' / 'laid out' methods, making long range forecasts with a good track record of been right. Only to have MB come up with some 'Piers Corbyn' style forecast with nothing laid out to be scrutinised, no logical proof/facts to back himself up, yet we must take this on the chin and not offer any negative feedback??

This whole saga kinds of reminds me when 'Alan Sugar' dealt with 'Stuart' on the apprentice, I kind of have come to the same conclusion. Im sure many of you know what alan discovered and said to him....

Well anyways if you want to be taken more seriously in the future MB you must:

• produce many accurate forecasts (with proof and facts that you've been correct)
• not just listen to praise and respond to that, also listen and respond to the people who have negative feedback, questions you perhaps don't like (youv'e ignored all TEITS posts.)
• Lay out your methods for everyone to see and scrutinise, that way no one can accuse you of 'guessing' (If your worried about your method getting stolen, get a patent on it first.)

Anyways hope I dont get slammed for having a different view, Im just saying it as I see it and backed myself up with facts.

Edited February 6, 2011 by Harsh Climate

[TEITS]

Folk can subjectively debate to the n^th degree whether the Storm Forecast was a hit or a very wild miss, or whatever in between; as already stated, my focus now lies in objectively researching the reasons behind the variances.

Finally, I'd also like to say many thanks to Coast, the other moderators and everyone who positively contributed to this thread.

Any chance of answering my question which to be honest is a fair question.

What I want to know is this. The GFS F.I charts can show massive differences between runs especially from around +288 to +384. This difference can sometimes be as great as a 950mb LP being replaced by a 1040mb HP even though at +144 the model run appears identical. Now I say appears identical because it actually isn't. The small changes earlier in the run is what causes the larger changes in the latter part of the run. This is where the chaos theory is taking effect because this difference becomes bigger and bigger as the model continues through the run. Now this is the whole reason for the ensembles. The starting point is slightly tweaked on the ensembles to try and replicate the chaos theory.

So as you can see from the above this is a major challenge for anyone trying to produce a detailed long range forecast such as yourself. However these models have an incredible amount of data inputted into them and the sheer amount of calculations required is something the human brain could never cope with. So again I ask how does your method take this into account?

One thing I will say if I was trying to predict the weather on a specific date here is what I would do. Say for example I tried predicting the weather on the 1st March 2011. I would look at the archives for the past 100 years and attempt to draw the mean of all those charts. However drawing the mean would be difficult because the archives for those past 100 years would show all different types of weather patterns but in my opinion it is the only method you can use.

[Coast]

Well it’s time for me to get off this fence, the splinters are excruciating! The forecast period is over and I have looked at the actual data and charts from each day and compared it to MB’s Great North Sea Storm forecast. Here is my personal overview.

Unfortunately the essence of the forecast was overshadowed by big headlines and some specific major events happening in areas adjoining the North Sea, reinforced with very high confidence in the outcome. As I have stated elsewhere on this thread, had Nick gone for any other period of relatively benign weather or something he could call without such spectacular overtones, then many people would have looked deeper into the prediction to analyse the substance. But Nick’s method predicted a Top 5 storm event was coming to the North Sea during a set period and he called that as he saw it.

If he had predicted something a little more general, say gale force winds across Scotland and the North during the later part of the first week of February, then we would have a hit on our hands, but that wouldn’t fill 37 pages of a thread or thousands of hits on YouTube.

Trying to put aside those headlines is difficult as although people on this forum like to drill down deeper into any weather forecast and understand more, ‘Joe Public’ is generally only interested in questions such as ‘can I have a BBQ on Saturday with a degree of confidence in fine weather?’ or ‘will I get snow at Christmas?’ or ‘will my house be flooded in two years time?’ If this could be done days, weeks, months or years in advance, they would all be extremely happy with those sorts of headlines, not the detailed substance. Unfortunately in this forecast we didn’t encounter many of the forecast events. We certainly had some high winds in Scotland but these were the result of a slow moving Atlantic low-pressure system born in Canada, which was not on this forecast but did show up in the models in the usual timescale. Winds over the period were higher in the UK, but again were on the whole from the West and South West and specifically part of the Atlantic system, not something derived from ‘The Great North Sea Storm’. If this were all that was being judged, I would say it was a miss.

Looking at the daily forecast maps, wind graphs, wave data etc and comparing to Nick’s version, a larger number of the forecast pressures, isobar spacings, fronts, directions and sea/river levels did not correspond with the actual daily analysis in either the location or the magnitude. Of course, I have not done the in depth research and analysis that RJS has carried out and as I am not one of the many knowledgeable members on here, I have just overlaid maps to see what lined up and what didn’t and added live data from sources in and around the North Sea to demonstrate the actual – maybe that was too simplistic and not a scientific approach? My apologies if this seems to ‘dumb down’ my comparison, it’s just my layman’s way of judging the success or otherwise. Using these criteria, I would also have to say that the forecast has gone astray for me.

Taking on a detailed weather forecast with (as Nick admits) only a year of study of the science was always going to be a tough task. I said from the beginning that I wasn’t interested in what his actual method is – and I still take that position. It’s Nick’s idea and he can do with it as he pleases and if there are sensitive commercial, personal, experimental or other reasons that prohibit him from sharing it, then so be it.

However, by not sharing the methodology with forum members and the weather community in general, we cannot help him any further. We can’t judge or analyse the method or how its applied and any benefit to predicting long or short range weather.

Specifically we don’t know if this forecast has generally missed the mark because:

• The method was incorrect
  
• The method applied to meteorology was incorrect
  
• The method using historical meteorological data was incorrect or
  
• The understanding of meteorology was incorrect
  

I’m sure Nick will continue with his studies. If I were him I would do some more work on the weather and experimental forecasts, see what the results are ‘behind the scenes’ and then come back with revised ideas and a new prediction at a later date - with a more tried and trusted product. It looks like only Nick will know which of the four points above have contributed this time. I also suspect that picking periods of weather with something a little less stunning showing in the billing may be the way to go, as people can concentrate on data rather than drama!

Thanks to everyone involved in the discussion for not letting it spiral too far out of control and to Nick for taking the time and effort to share this forecast with us - please keep contributing to all areas in the forum. So its back to looking at traditional methods for the foreseeable future personally, and that includes the roller coaster of the model threads and all that FI brings (or doesn’t!)

Better then the models? – Not on this occasion for me.

[johnholmes]

I have kept out of any discussion but read each post from the outset and I would say the summation by Coast is pretty much what most folk on this forum would come to if we took the time.

To me a miss also.

[TEITS]

I have kept out of any discussion but read each post from the outset and I would say the summation by Coast is pretty much what most folk on this forum would come to if we took the time.

To me a miss also.

Morning John.

I have a question based on your many years of forecasting. Do you think the type of detailed forecast MB is attempting will ever be possible? My view is we might be able to predict the general weather patterns but we will never be able to produce an accurate, detailed long range forecast. I believe this will be the case even in 50-100yrs time. At the moment I feel the limit of producing accurate, detailed forecasts remains around 7 days i.e countryfile forecasts.