Better Than The Models ?

[johnholmes]

Morning John.

I have a question based on your many years of forecasting. Do you think the type of detailed forecast MB is attempting will ever be possible? My view is we might be able to predict the general weather patterns but we will never be able to produce an accurate, detailed long range forecast. I believe this will be the case even in 50-100yrs time. At the moment I feel the limit of producing accurate, detailed forecasts remains around 7 days i.e countryfile forecasts.

hi

I have no knowledge of the method MB has used, nor for that matter with say RJS and BFTP, so my comments are restricted to 'standard' forecasting, be it short time or longer scale, such as UK Met out to 30 days. Beyond that then its a 'new' development even for UK Met who are doing pretty intensive work on trying to find ways of utilising current knowledge about sun effect, earth wobble, down to NAO developments to try and improve seasonal forecasting.

Out to 30 days and I don't believe it will ever be possible to give actual dates of particular events, beyond say 7-10 days, possible 2 weeks at the most. A guide to a period of several days MAY become possible rather than early or late in the period as the forecast will sometimes say now.

For seasonal then it has to be on an even coarser scale in my view. A general indication of above/below normal values for the major meteorological parameters, temperature, rainfall, sunshine, wind strengths etc. Also an indication of the likely period for these departures, 'at the beginning of the period, month etc'

Like I say this is based on 'standard' forecasting methods, allowing for them to evolve over several decades. As to any of the other methods then I have as I said little or no understanding. That said some of the efforts by RJS do seem to show promise but its way above my understanding of his methods.

Edited February 7, 2011 by johnholmes

[casparjack]

I think what TEITS suggested above in looking back at 100 years of archive as a starting point for such a long range detailed forecast, whilst perfectly valid as a starting point, highlights how misleading the widespread use of the 'mean' as a benchmark for weather conditions at any time of the year is.

My point is that you so often hear the media forecasts referring to 'average for the time of year', but when you think about it, this is a pretty meaningless statement - broken down, what it means is the temperature on any given day might be close to the average of a data series for that day, week month whatever you choose over a period of 'n' years.

Now within that sample there will be large variations from the 'mean', resulting from exceptional synoptic conditions which in themselves might be very rare due to the many variables required to be in place to generate such resultant conditions. These extremes are 'diluted' by calculating the mean.

My argument here is that for such data analysis, 'the mean is meaningless'!!

One could argue that 'modal' analysis would be a better indication of the most common conditions at any time of the year - though I appreciate it's possible to have similar temperatures as a result of many different synoptic types.

What might be more valuable is a method of illustrating that at most times of the year and particularly during the winter period our weather is extremely disturbed, especially when the Atlantic has its traditional way - we can experience huge ranges in temperature in a 24 hour period and the weather conditions associated with these are equally diverse.

I don't have a perfect solution, but what I'm trying to say is that in my opinion the mean is neither useful as a forecasting tool or really as a comparator for synoptic conditions.

Probably not the right thread for this, but this prompted something that's been on my mind for a while...

[lorenzo]

Well I thought this ecercise was entirely enjoyable - esp with the limpet high in January !! = well done MB for putting it out there - yes, it is brave of you.

In respect i believe IMBY I had a mini tornado - window on car popped and fence and fauna on same path fell, although that was majorly ue to the massive PV acting as a reinforced roof to atlantic interjection and throwing things heavy west..

In respect of qualification of anything else - sorry miles off - I will re-iterate my earlier post- your method born of analysis of wheat trade futures and cycles leaves me as purely sceptical. As SM qualified the models used to forecast weather have heavy physics behind them, just look at VP post on what is statistically an Outlier to see this..

I appreciate your work and effort - just be more general next time and give some clues to the method - that will attract wider interest, wider audience, wider respect...

You cold list 5 factors or 40 factors in your summation - it wouldnt matter as 12,000 weather enthusiasts would all have differing opinions on the lot of them ! Don't stop forecasting just give folk some clues...

[Roger J Smith]

I took some time to think over the discussion of chaos theory and investigate further.

No arguments about the validity of chaos theory per se. I think I was reacting more to the layman's inevitable interpretation of the concept that very long-range (past 10 days if you like) forecasting reduces to chaos. This may not be what is really meant by any of the posters here.

I think we all understand that getting the standard accuracy degradation curve to improve will take some kind of breakthrough in one of two areas, either better super-computers, or a theory that can accurately place pressure systems from some theory rather than an extension of what's known at forecast time. VP, I don't read into your comments any "proof" that weather systems must exhibit chaos over time, rather that chaos theory provides the best understanding we have today. The analogy of the "infinite coastline" seems obscure to me -- I think most laypersons would accept a "good fit" approach of saying that the coastline of Great Britain was (just tossing out numbers) 1578 kms if one went point to point and excluded any cases less than 0.1 km. This would be as accurate as today's forecast, if you see what I'm driving at, so that giving it as an example of chaos theory would really undermine belief in even what is already trusted in meteorology (generally speaking).

I don't think anyone active in LRF research is striving to create in this decade or even this lifetime a result that matches 24-hour model forecast accuracy. I think the search is for something better than what I just demonstrated in this thread, which got very little comment either way, but did produce a correlation above 0.4 at 22 days. My effort published here in December 2005 from 30 days had a correlation fairly close to 0.7 and some other case studies that I have done scattered around where the GFS or ECM normally perform at day 7-8. By the way, this MB test case actually gave the two main models a chance to show off their better sides, as they tended to score better than their average correlations by about a two-day increment (i.e., day 8 performed like day 6, etc).

So the question then becomes, can this be done? Can we get from what seems almost random today, to what might strike people as "predictive" in the same way that people respond to the 3-6 day portion of the models? We all know those are subject to change and fine-tuning, but who on a weather forum would say "I only look at 48h and inside that time frame, otherwise it's all random." ?? Well sometimes people say that when they get frustrated with model flip-flops.

I remain absolutely convinced that such progress can be made, but I need to assess over the next year (while still working at some level near peak) what needs to be done with my own research results, because I have to make a rather difficult choice here. Assuming I am going to have to keep working alone on my own budget (not quite as demanding as it sounds now that pensions are in the picture) which of the following courses of action should I follow?

** step back from active forecasting and any distractions from research, and just keep trying to publish, from the basis of work already finished and anything I can add in 2-3 years. -- the end goal of this would be to encourage limited recognition that would lead to parallel research (other workers number crunching other regions, basically, as the global challenge is frankly beyond one man's capability in real time).

** take a sort of mental sabbatical and see if I can identify what unrelated field of research needs to be integrated with my material to overcome the accuracy deficit implied by correlations in the 0.4 to 0.6 range.

** give it 2-3 more years on the current set of assumptions and short-range efforts (whether people here realize it or not, I am very busy on three other forums, and I have short-range forecast commitments that anchor my research day, sometimes in the sense of getting in the way of it, other times helping with it.

So that's the kind of last ten minutes of the match I face (with any luck there will be injury time). Some would say I got red carded early in the first half and should be in the stands watching.

The thing that does keep me going forward with this life-long effort is, I know there's something to it, and I don't see anything particularly out in front, although the teleconnection approach is obviously a big part of the "set-up" half of this puzzle. For anyone who didn't get this key piece of the puzzle earlier in the thread, the puzzle has two basic parts -- set-up and energy level. MB has just demonstrated that energy level is important, but if you don't have set-up worked out, then energy level will be sort of like installing a car's wheels at an angle to the frame.

My effort, it would be fair to say, missed somewhat on both counts, and resembled a car with a flat tyre and engine problems.

If anyone wants to have a go at 20 March 1200z from any significant time before the 16-day GFS on 4 March, let me know. I am working on a map for that date because of a large energy peak diagnosed in the research. I may produce two maps, one for the UK region and one for eastern N America where I think my results are somewhat more advanced.

All for now, but further discussion of the random nature of weather events past day 6 would be interesting.

Edited February 15, 2011 by Roger J Smith

[VillagePlank]

... but further discussion of the random nature of weather events past day 6 would be interesting.

Of course, the weather isn't random, so, presumably, it must necessarily be deterministic with the consequence that if we could know everything there is to know, then we could entirely predict the weather at any time scale.

My own view is that, generally, chaos theory is an observation that man-kinds efforts to functionally decompose nature into its subsequent laws and model it are by it's very nature limiting. That is to say that we cannot possibly know everything so the approach that breaks up nature into laws will give progressively less returns. And, by extension, since the existence of sensitivity to initial conditions when we model systems on a computer implies non-determinism, even though in the real-world, things tend to be perfectly deterministic, forecasting weather out past 7 days is, I think, NP hard, which in turn strongly suggests that there is not enough time between now and the heat-death of the universe - even if we could get hold of all the data - to compute an answer to the problem.

As inquisitive creatures, then, we must try an alternate approach; which is why I often find strange, eclectic, and normally novel approaches to forecasting weather interesting. Of course, MB seems to delight in keeping secret his method - at least that is my take on things - which leaves me none the wiser to whether I have, once more, had the wool pulled over my eyes; since I can get my two year old to scribble down some lines on a map of Europe and then claim some connection to ancient cultures who knew of all sorts of facts to do with nature but failed, of course, to extend the human-life span, which has only really happened in the last 200 years of so.

That said, I strongly believe - and, yes, this is an article of faith - that some method is discoverable; it may well lie in some abstract of already known physics, it may lie in a new discovery. I don't know. But I'm certainly looking out for it.

Edited February 17, 2011 by VillagePlank

[La Bise]

Although we diverge at opposite angles when it comes to the philosophical core of the issue (deterministic vs chaotic), it has been a pleasure reading your interventions VP. Good luck in your quest, even if it is not achievable it's best to give it a jolly good go if so is your inclination (I have my own quests, almost certainly futile, but this what defines us as human beings, our capacity to overcome rationnal to follow our dreams).

Wonder what happend to MB, maybe on the blower with the Wise Ancients asking what went wrong...?

[Weather-history]

I think whats happened over the last few days is an example why I don't believe you can make precision forecasting so far out. Even now after so many runs from the models, there is still some doubt about what exactly is going to happen. How would such far ahead forecasting would cope in such a situation? It would struggle, IMO. The best you could get it down to is probably a battleground scenario and that is about it. I don't think you could say snow on the 5th, rain on the 6th etc. Its these scenarios where small random elements can have a huge impact.

[La Bise]

I've fount very interesting to see how people dealt with the situation on the model thread. Despite a)repeated comments by the Met guys that models do struggle with that kind of setup and B) the events of November/December when another block situation proved very hard to forecast, there seems to be a large consensus that modelisations do really represent a fair picture of what might happen. Which might be right in some situations but in this case, some much depends on tiny variables that human input, the capacity to forecast rather than interpret model is very much a must. That's for me is the main difference between enthusiastic amateurs and the Met chaps, that capacity to call upon human judgement much more effectively.

[VillagePlank]

Although we diverge at opposite angles when it comes to the philosophical core of the issue (deterministic vs chaotic), it has been a pleasure reading your interventions VP. Good luck in your quest, even if it is not achievable it's best to give it a jolly good go if so is your inclination (I have my own quests, almost certainly futile, but this what defines us as human beings, our capacity to overcome rationnal to follow our dreams).

Wonder what happend to MB, maybe on the blower with the Wise Ancients asking what went wrong...?

The classic example of finding patterns in non-linear dynamics is the pendulum. We can use such things to tell the time, so it is highly reliable, and predictable at some level of abstraction. A pendulum also has other highly unpredictable properties.

The key, then, is finding the suitable level of abstraction with regard to forecasting weather. I think that that exists in that very large and very grey area that exists between weather and climate.

Edited February 17, 2011 by VillagePlank

[MurcieBoy]

For those that have taken a keen interest in this thread and also for completeness, here's the appraisal video to the Feb 2011 Storm forecast

[kpm62]

For those that have taken a keen interest in this thread and also for completeness, here's the appraisal video to the Feb 2011 Storm forecast

I think thats a fair analysis.Like you state MB there were some accurate comparisons that no sceptic can deny.With regards to your future forecasts, have you seen any other extreme weather for the UK this winter?

[johnholmes]

For those that have taken a keen interest in this thread and also for completeness, here's the appraisal video to the Feb 2011 Storm forecast

Well put together but I do think you concentrated on the wind strength rather than its direction which is what caused so many ears to prick up when you produced your forecast.

Most models, granted not the GFS charts you showed did not, but the longer term 'synoptic' type charts from their initial idea of a 'mobile' rather than a blocked Atlantic were suggesting a return, at least for a time, towards the end of January/early February of this mobility.

Drivers such as the AO and NAO down to the 500mb anomaly charts, in their rather different time scales, did show this.

As to what method you use then I have no understanding of it but do suggest you explain it, show a succession of forecasts at a similar time ahead, and let all judge you on several outputs. The more the better. If you succeed in a score of over 50% over say 10-12 predictions then you can begin to feel you have something going for you. Another idea, not dissimilar to Rogers' input is for someone other than yourself to assess you. I know from my own experience that trying to be objective about ones own forecasts is incredibly difficult.

good luck anyway.

[MurcieBoy]

I think thats a fair analysis.Like you state MB there were some accurate comparisons that no sceptic can deny.With regards to your future forecasts, have you seen any other extreme weather for the UK this winter?

Thanks.

Been busy with other work for the last few weeks. Did take a squint from my rough notes from last summer, I have something windy/stormy in the UK for August and November this year. Will look at these more closely (and whether there is more extreme weather in the UK for the rest of the winter) when I have a bit more time.

Got a couple of extreme weather events in the US that I might look into further and publish some forecasts.

At the moment, I am trying to reflect a wee bit on the Feb 2011 storm forecast. I think from one perspective it may be pointless churning out more forecasts without squeezing out what I can learn from this one.

Edited February 26, 2011 by MurcieBoy

[MurcieBoy]

Well put together but I do think you concentrated on the wind strength rather than its direction which is what caused so many ears to prick up when you produced your forecast.

Most models, granted not the GFS charts you showed did not, but the longer term 'synoptic' type charts from their initial idea of a 'mobile' rather than a blocked Atlantic were suggesting a return, at least for a time, towards the end of January/early February of this mobility.

Drivers such as the AO and NAO down to the 500mb anomaly charts, in their rather different time scales, did show this.

As to what method you use then I have no understanding of it but do suggest you explain it, show a succession of forecasts at a similar time ahead, and let all judge you on several outputs. The more the better. If you succeed in a score of over 50% over say 10-12 predictions then you can begin to feel you have something going for you. Another idea, not dissimilar to Rogers' input is for someone other than yourself to assess you. I know from my own experience that trying to be objective about ones own forecasts is incredibly difficult.

good luck anyway.

Thanks, took a wee while to gather the info and to put the thing together. The Satellite Images sequence alone took ages to download (there were almost 500 of them!).

Each to their own as to how to evaluate; yes, always best to have an objective evaluation.

For me, if there were no really strong winds/very strong Jet Stream in the target area and in the time specified that would have been the biggest failure.

[Timmytour]

Mb

Not a forecast that will have me singing your praises particularly....but not so far out that I'm no longer interested in any future forecasts you have.

I told my 8yo son there was going to be a big storm at the beginning of February...he in turn told some of his schoolmates and came to an arrangement with one of the girls that if there was one she would have to do whatever he said for one day and vice versa. I didn't want to enquire too much as to what exactly they would be asking each other to do....but suffice to say that my son was happy as he was proved "right".

[MurcieBoy]

I told my 8yo son there was going to be a big storm at the beginning of February...he in turn told some of his schoolmates and came to an arrangement with one of the girls that if there was one she would have to do whatever he said for one day and vice versa. I didn't want to enquire too much as to what exactly they would be asking each other to do....but suffice to say that my son was happy as he was proved "right".

My, that has made me smile! Brilliant!!

[Coast]

I think from one perspective it may be pointless churning out more forecasts without squeezing out what I can learn from this one.

Yes

The thing I have learnt is that putting dramatic music onto a video, then showing an Atlantic storm with clips of action in Scotland when we were all supposed to be looking towards Holland for one of the TOP 5 storms of the last 300 years + trying to use that as an example of the 'Great North Sea Storm of February 2011', would surely get you in trouble with Trading Standards if you were to sell it (let alone the TV company copyright for the weather clips!!!) :lol:

I'm sure many people could have predicted this follow up video, comments and sleight of hand that turned around a forecast that was ultimately wide of the mark on the major points. But I am disappointed that you haven't been more prudent and circumspect in your self-analysis and I'm afraid this has pushed an interesting idea into Piers Corbyn territory now and I think I'm going to leave it there.

As I said at the beginning, if you hadn't of been so specific in the events, magnitude, direction, location and timing, you may have convinced me that even though throwing a head, you meant it to be a tail.

[MurcieBoy]

Yes

The thing I have learnt is that putting dramatic music onto a video, then showing an Atlantic storm with clips of action in Scotland when we were all supposed to be looking towards Holland for one of the TOP 5 storms of the last 300 years + trying to use that as an example of the 'Great North Sea Storm of February 2011', would surely get you in trouble with Trading Standards if you were to sell it (let alone the TV company copyright for the weather clips!!!) :lol:

I'm sure many people could have predicted this follow up video, comments and sleight of hand that turned around a forecast that was ultimately wide of the mark on the major points. But I am disappointed that you haven't been more prudent and circumspect in your self-analysis and I'm afraid this has pushed an interesting idea into Piers Corbyn territory now and I think I'm going to leave it there.

As I said at the beginning, if you hadn't of been so specific in the events, magnitude, direction, location and timing, you may have convinced me that even though throwing a head, you meant it to be a tail.

I am reluctantly thinking that whatever I did Coast, you would criticise and try to ridicule. If you are able to be objective/balanced, perhaps you should read all the text on the videos more closely; rather than being so selective, so as to promote a negative angle with a view of trying to mock another human being.

[Coast]

I am reluctantly thinking that whatever I did Coast, you would criticise and try to ridicule.

It hasn't worked for me as I have said in this thread and shown in as much detail as is available to me, with analysis of your daily charts and predictions to the actual outcome data. I'm sorry, it's my opinion on the outcome - I waited until it had passed and felt I gave the comparison of your forecast to the actual result, as best a shot as I could in my simple way.

[MurcieBoy]

... dramatic music onto a video......Trading Standards..... sleight of hand .....pushed an interesting idea into Piers Corbyn territory.... you may have convinced me that even though throwing a head, you meant it to be a tail.

There is no mockery or ridicule implied or actual. [before edit]

Yeah, right!

[johnholmes]

Yeah, right!

come on MB

this is YOUR headline that you posted

the Great North Sea Storm of 2011

the response has to be what Great North Sea Storm of 2011-simply accept it did not occur as you predicted it would.

strong/gale force winds did occur in the north west of the UK.

You will gain little respect I am afraid if you cannot accept when you get a forecast wrong. I learnt that decades ago.

[PB22]

come on MB

this is YOUR headline that you posted

the Great North Sea Storm of 2011

the response has to be what Great North Sea Storm of 2011-simply accept it did not occur as you predicted it would.

strong/gale force winds did occur in the north west of the UK.

You will gain little respect I am afraid if you cannot accept when you get a forecast wrong. I learnt that decades ago.

Well said John.

I watched the first 30 seconds of the video then gave up as it was clear that it was heading into, well, as Coast put it, Piers Corbyn terrority. So any respect soon went out of the window.

The forecast was totally wrong, that's the fact of the matter. Yes there were strong winds and gales, but not in anyway as indicated by the original forecast. Therefore the forecast was wrong. It's like buying a can of baked beans to open them and find they are in fact green beans.

[MurcieBoy]

come on MB

....simply accept it did not occur as you predicted it would.

Of course I do, I am the first to admit it!

If you read the text in the Appraisal video it states:

I don't know if I could have put it more candidly or bluntly that it was wrong!

[La Bise]

You did, a little frame at the end after an endless sequence of storm pictures with dramatic music to hammer the (erroneous) point again and again and again.... Stop playing dumb MB, you know perfectly well what you are doing with that video, your "candid appraisal" sounds about as sincere as an insurance salesman pointing out to the small prints after refusing a claim...

What bugs me is the way you whine about Coast comments when the guy did his outmost to be fair to you all the way up to the end of your forecast period and put lots of work in it.

[johnholmes]

Of course I do, I am the first to admit it!

If you read the text in the Appraisal video it states:

I don't know if I could have put it more candidly or bluntly that it was wrong!

sorry to keep going on about this, the longer serving folk on the forum will know I am a bit of a stickler for getting quotes correct.

You said North Sea, that did not occur. You got it wrong, and your video seeks to prove you got most of it right.

Please stop insulting the intelligence of members on this site.

Prior to this most of us were prepared to give you a fair hearing, a few failed to do that for you, the majority did wait to see what happened. Principle amongst them being Coast who, you thanked him at the time, kept showing a whole variety of data, UK Met Fax charts superimposed on your predictions being one sequence. Now he makes his honest comment and you get upset.

You cannot post ideas/forecasts with such bold statements and not expect criticism.

Edited February 28, 2011 by johnholmes