How forecasts are created

[Ken]

For anyone interested in the sorts of things that are taken into account when public forecasts are created, below is an example. It's a forecast discussion created by the US National Weather Service office in Upton, New York for that region which gives the reasoning behind the public forecasts for the info of other forecasters and adjoining forecast offices. It's typically updated at least a few times each day (freely available at: http://forecast.weather.gov/product.php ... hlight=off). The trend these days is to use automated systems which remove under/over-estimating tendencies from models, combine their outputs, weight better models more heavily and spit out a consensus forecast. These also give a good idea of how much uncertainty/confidence there is among forecasts and therefore, the probability of things occurring.

The Bureau here issue similar products internally called a forecast policy. They're normally created by a senior/supervising meteorologist in each state's forecasting centre and give overall guidance to other forecasters in that state. The style of the forecast policy itself varies from state to state e.g. Victoria typically uses "mudmaps" or diagrams which show what area will get what weather while QLD uses both a detailed text description and mudmaps. By the way, some of the abbreviations below include: RUC and NAM (Rapid Update Cycle and North American Mesoscale - both are models used in the US), QPF (Quantitative Precip - basically forecast rainfall amounts) and POP (Probability of Precip):

AREA FORECAST DISCUSSION
NATIONAL WEATHER SERVICE NEW YORK NY
1047 PM EDT SAT JUN 19 2010

.NEAR TERM /UNTIL 6 AM SUNDAY MORNING/...
BASED ON LATEST RADAR TRENDS SHOWING CONVECTION OVER S CENTRAL PA
WEAKENING...AND 18Z NAM AND 20Z RUC KEEPING REGION DRY OVERNIGHT
(AND BOTH THESE MODELS HAVING A FAIRLY GOOD HANDLE ON CURRENT
CONVECTION - 18Z GFS HAS A FAR TOO EXTENSIVE QPF FIELD WITH
ONGOING CONVECTION AND ALSO APPEARS TO BE SUFFERING FROM
CONVECTIVE FEEDBACK ISSUES SO HAS BEEN DISCARDED FOR THIS UPDATE)
HAVE REMOVED POPS FOR MEASURABLE PRECIPITATION FOR THE FORECAST
FOR TONIGHT.

WHILE THERE COULD BE POCKETS OF FOG OUTSIDE OF THE METRO AREA LATE
TONIGHT THAT REDUCE VISIBILITIES TO UNDER 3 MILES...THIS IS NOT
EXPECTED TO BE WIDESPREAD ENOUGH TO MENTION IN FORECAST AT THIS TIME.

UPPED SKY COVER A BIT BASED ON RECENT SATELLITE TRENDS SHOWING
CIRRUS BLOW OFF FROM CONVECTION OVER UPSTATE NY/CENTRAL PA
ENCROACHING ON REGION...PLUS HAVE SCT-BKN STRATUS IMPACTING THE S
SHORE OF LONG ISLAND FROM TIME TO TIME ALREADY.

FOR TEMPERATURES...ALREADY RUNNING COOLER THAN FORECAST...SO
ADJUSTED LOWS DOWNWARD...THOUGH WITH LOW LEVEL WARM
ADVECTION...WOULD NOT BE SURPRISED IF TEMPERATURES EITHER HELD
STEADY OR BEGAN TO SLOWLY RISE AFTER MIDNIGHT.

&&

.SHORT TERM /6 AM SUNDAY MORNING THROUGH TUESDAY/...
18Z NAM/20Z RUC WHICH HAVE GOOD HANDLE ON ONGOING CONVECTION
SUGGEST THAT THE PRE-FRONTAL TROUGH SHOULD CROSS THE AREA DRY -
HAVE UPDATED WEATHER AND POPS FOR SUNDAY MORNING TO REFLECT A DRY
FORECAST (A FEW VERY ISOLATED SHRA CANNOT BE COMPLETELY RULED OUT
SUNDAY MORNING...BUT WITH POSSIBILITY ASSESSED AT 10 PERCENT OR
LESS...HAVE NOT REFLECTED IN FORECAST).

DURING SUNDAY AFTERNOON AND EARLY EVENING HOURS AREAL CONVECTIVE
COVERAGE DEVELOPS THEN INCREASES ALONG AND AHEAD OF COLD FRONT
APPROACHING FROM WEST. THE COLD FRONT WILL MOVE ACROSS THE AREA
DURING THE AFTERNOON AND EARLY EVENING HOURS...CLEARING EASTERN LI
AND SE CT BEFORE MIDNIGHT.

THUS FAR THIS SEASON...SEVERE WEATHER HAS BEEN ISOLATED WITH
GENERALLY SUFFICIENT SHEAR...BUT MARGINAL INSTABILITY. FOR SUN
AFT BULK SHEAR WILL BE IN THE 30-40 KT RANGE WITH ML CAPES AROUND
1000-1500 J/KG. THERE IS THOUGH A MID LEVEL CAP AT AROUND 600
MB...WHICH IS ABOUT A DEGREE STRONGER IN THE GFS VERSUS THE NAM.
THUS...THE POTENTIAL IS THERE FOR SOME SCT STRONG...PERHAPS A FEW
MULTICELL BANDS/CLUSTERS. BEST INSTABILITY WILL BE ACROSS THE INTERIOR
WITH CELLS WEAKENING SOMEWHAT AS THEY GET ACROSS EASTERN LI AND SE
CT DUE TO THE MARITIME INFLUENCE.

[Karl Lijnders]

Your a legend Ken. Thanks so much for this useful info!! Great reading!!

[Anthony Violi]

Agree with that Ken, since you have joined the board we have had access to stuff that we previously wouldnt have.

A real eye opener for all of us, keep it up as i really look forward to seeing your posts.

[Ken]

No probs. I forgot to add that many weather agencies are also using networks of webcams these days, both private as well as dedicated internal ones to get a visual idea of things like early onset of fog in different areas.

[Karl Lijnders]

So in a case where we have seen in Melbourne:

Forecast is the night before issued at 4:20pm

Thursday July 2nd 2022

Melbourne: Showers developing during the morning with the risk of a thunderstorm. Strong and gusty northerly winds tending southwesterly during the day. 17C

------------------------------------------------------
So we all go to bed and the wind is raging outside etc. Typical of a winter evening ahead of the front. But overnight the rain comes in a bit earlier and starts to stack up on the ranges to the NW. Potentially it could stay there until the front comes, but as we get towards 7am on the 2nd of July the rain descends off the ranges. The new forecast came out at 5am that day and has not been altered.

At what point do they decide to change or update the forecast? Does the forecaster have to jump through hoops to do so or has free reign to go ahead and change the forecast?

[Ken]

It's usually done on a case by case basis and depends on the situation at the time Karl. If new or updated data comes in which the forecaster thinks justifies a forecast change, the forecast should be changed. The tricky bit is deciding what warrants a change to the forecast. In some situations, it's clear-cut. In others, the forecaster has to decide whether the new data is suggesting a real divergence from the forecast or whether it's just some temporary trend that's likely to disappear (which I've seen happen often). So there's a big risk of a knee-jerk reaction whenever something appears on radar, satpics or data that's not quite what was forecast.

So in your example, if that rain stays as real rain as it closes in on the city (the correct definition is basically steady liquid precip from stratiform rather than cumuliform cloud and usually widespread/uniform), past experience shows that it probably won't disappear as it gets closer due to rainshadow effects, modelled and/or analysed data suggests things are diverging from forecast, etc, I'd be surprised if the forecast wasn't updated. There can also be a bit of a delay in public forecast changes coming out because some time's taken up looking at the data to confirm that it's a genuine change. Then the processing and issuing of the forecast changes have to be prioritised because they're more critical to certain sectors such as the aviation industry than others (they need to know very specific info such as timings and changes to cloud base, vis, etc whereas the general public don't). Having said that, I've seen some amusing "non-changes" to forecasts in the past.

[crikey]

Ken
is all forecasting done by pressing buttons on various weather forecasting software . If the software is not getting good reliable results is the software dumped.
I notice the forecasters on ABC always refering to various software forecasting models. Do they really need them. I noticed on their site there's a new one coming.
I just shudder at the thought of their experiment being trialled on the Australian population
BOM have NOT got the wind direction right in central vic for a number of days if you can't get the wind direction right then nothing else falls in to place correctly. I often keep forecasts from newspapers to keep the synoptic charts for my own interest and look back on the forecast for the day and wonder at times why the anomalies?
. I never read the 5 day forecast anymore it means nothing.
Timing and content is often out by a mile.
, 3BO radio station often overrule the BOM forecast and make polite remarks that the forecast is a bit off or won't happen.
I have become cynical of the forecasting ,disillusioned. If you were to tape record the different forecasters on ABC radio and compare various newspapers it is often so confusing often very inaccurate discourse of the weather
Things like forecast a few mm and then we get the floods, warnings given out to Melbourne 10 minutes after the storm has passed etc.
If they were audited they might be found wanting. I can't help feeling it has something to do with a heavy reliance of computer modelling and the lack of forecasters with good brains to have the freedom to make timely over rulings . Something amiss in the organisation?
They treat these models as though they were Gods.
BOM got it perfect for YASI.. but it gets a bit sloppy elsewhere and has been downright pulled out of the hat for our area for quiet some time
Maybe the problem is they have no competition and have the monopoly on forecasting.
Anyway its good business for independent forecasters like on AWF. People come looking for something that they can comprehend and vaguely trust.

[Lily]

Ken, you're a deadset legend, thank you so much for everything you post here. And you're still my favorite Awesomesauce

I do have to agree with crikey re: SWW's for storms that are well past or overhead. In your experience, what causes the delay?

BTW, my question is not "BOM bashing" by any means, I'm genuinely interested and I understand that like many organisations, you're (collectively) bound by the rules and regulations of your employer and can only operate within established parameters.

[Jake Smethurst]

This is an interesting question regarding Severe Thunderstorm Warnings, however the process of how they are created is very interesting, and I would love to have similar software to use with the AWF. However, no doubt, it would be extremely expensive, but the process of the warning creation looks rather simple from what I have seen.

I remember from when I was doing work experience at the Victorian Regional Office with the meteorologists, they showed me how to create the warning, and it generally does not take very long.

As I mentioned, the process is very interesting, and something that I would love to do.

[Ken]

Ken
is all forecasting done by pressing buttons on various weather forecasting software . If the software is not getting good reliable results is the software dumped.

This misconception comes up all the time in debates like this. No official forecast is completely automated without the ability for human input. Vic and NSW use a system called the GFE (Graphical Forecast Editor) which takes model data and converts it to text forecasts for the public. Forecasters can, and often do manually edit the data depending on the situation before the forecasts are released. In other states where GFE hasn't been rolled out yet, forecasters look at the models and together with their knowledge of meteorology, manually create the forecasts. I can't speak for Vic/NSW but my understanding is that with GFE, there's been some issues with consistency of forecast data on small scales (especially when local effects cause a big difference in temperatures over a short distance) as well as one of its components that convert the data into text (which sometimes results in unusually-worded forecasts). You also have to remember that much more of a forecasters' time has to be devoted on reviewing 7 day forecasts for all these smaller towns which never used to have 7 day forecasts.

I notice the forecasters on ABC always refering to various software forecasting models. Do they really need them. I noticed on their site there's a new one coming. I just shudder at the thought of their experiment being trialled on the Australian population

It's not an ABC thing. Models are used all over the world by national weather agencies and private weather companies as the main form of guidance for forecasts. Without them, it's impossible to forecast weather to a decent level of accuracy, especially beyond a few days. There are some situations where forecasting rules based on the past behaviour of certain types of weather systems and principles of meteorology can give you an idea of what they're likely to do over the next few days but only a general idea. There's a long-standing attitude among many people that technology is never as good as "good old-fashioned human" methods just because it's technology, but that's simply not the case when it comes to forecasting. One of many examples is consensus forecasts from a combination of models having been shown to outperform human forecasts for minimum temperatures. In reality, it's the combination of models and human input (which provides the checks and balances) which still gives the best results overall. It's still a case of you can't have one without the other if you want to make realistic forecasts. Having said all this, I don't know what the actual stats are on Victorian forecast accuracy these days being in QLD (I've heard many stories though), but the issues I mentioned earlier in this post would be something to consider.

As for warnings, I agree with you and some others here. My personal opinion is that it would be good to have "heads-up" type advisories issued much earlier than the actual warnings are issued now. Even though forecasting when and where severe thunderstorms will hit is tricky when conditions are marginal, I think there's merit in issuing advisories earlier when guidance shows they're very favourable in certain areas up to a day or two before the event. "Heads-up" flood watches are currently issued in some states but I think this idea could be expanded to cover other weather types such as widespread severe thunderstorm outbreaks. This is nothing new though - it's been done in other parts of the world including Japan, the US and the UK for some years.

Maybe the problem is they have no competition and have the monopoly on forecasting.

Competition or having a monopoly on forecasts doesn't really come into it. Being a national Met centre and not a commercial organisation, the Bureau doesn't, nor is allowed, to profit from public forecasts. This is the same with other national met centres around the world which are federal government agencies. National met centres are required under their charter/government, to be the source of official weather forecasts and warnings regardless of how many private weather organisations also choose to produce their own forecasts.

Anyway its good business for independent forecasters like on AWF. People come looking for something that they can comprehend and vaguely trust.

If you want to make AWF truly reputable for forecasts, I'd recommend creating a system to keep running stats on exactly how many *incorrect* forecasts as well as correct ones were produced, what tolerances were used to decide whether a forecast temperature, rainfall, etc was right or wrong and make this accessible to anyone who wants to view them. The same thing is done for national met centres as well as all models where forecast verifications and skill stats are run every day/week/month/year to keep track of how well human and model forecasts are doing.

[Ken]

This is an interesting question regarding Severe Thunderstorm Warnings, however the process of how they are created is very interesting, and I would love to have similar software to use with the AWF. However, no doubt, it would be extremely expensive, but the process of the warning creation looks rather simple from what I have seen.

The software system automates a lot of the process so it simplifies things. It uses info from TITAN (a system that uses radar and GPATS lightning data to automatically identify and track past and future thunderstorm movements) and lets you select all sorts of options to generate the severe thunderstorm warnings. Have attached a screenshot below. Sorry about the size.

Lily: re delayed severe thunderstorm warnings, storms have to meet one of certain criteria to be called severe thunderstorms e.g. winds > 90km/hr. Some storms can look severe on the public radar images but don't show severe characteristics on 3D radar imagery which lets you cut "slices" through any part of a storm at multiple radar elevation angles to show wind and rain intensities. Software algorithms can also display estimated hail size in storms. Also, some severe storms can suddenly pop up "out of nowhere" with very little warning - individual thunderstorms not part of an organised line can be very hard to forecast for. But as mentioned in my other post, I still think it would be good to have more advanced warning of widespread severe thunderstorm outbreaks in cases where conditions are obviously favourable for them. And thanks for the awesomesauce comment. It still sounds saucy. I don't think I've ever had my own groupie on here before.

TIFS - ML 0801z 6th Oct 10.png (237.01 KiB) Viewed 6473 times

[crikey]

KEN
Thankyou for your most informative reply

Talking from the everyday view of people like me who pick up the herald sun and listen to the radio and TV stations , forecasts are something that just sort of sit in the back of your mind and over time you build an impression.

I agree that software simulations have a role in forecasting and the everyday mums and dads like me never follow them. we only read the end product that rocks up in the media. Maybe the issue is presentation.
Do you have an evaluation section that focuses on the consumer of this information ..as a means of evaluating forecasting ?
For example for the Nth central region, we have 4 statements for the daily forecast
temperature : which is rarely wrong!
wind direction: which is generally right with the occassional very incorrect
cloud: terminology is prone to misinterpretaion. ( partly cloudy?)
precipitation: showers , rain , thunderstorms etc

They are often written in the definitive or diagrams in the definitive: Picture of thunderstorm with rain laden cloud

cloudy
sunny
Rain
Thunderstorms
To me that means it will rain and it will thunderstorm over me and of course .when it doesn't l am disillusioned
If l see a sunny icon and it starts to get cloudy l get disillusioned etc

Maybe it would be more realistic to use terms like
chance of rain
likely showers
wind range south to SW
cloud: 20 to 80 % cover
showers 2 to 5mm
Rain 50 to 100mm
possible thunderstorm
Watch out for possible severe storms
temp: daytime high expected between 10 to 15 degrees

Weather forecasts are forecasts and not definitive so why write any of them in the definitive?
It leaves BOM open to criticism

Some of the symbols used on the weather forecast maps are misleading.
When you look at a picture of a grey cloud with little rain drops under it my mind says it is going to rain, however the symbol may refer to showers etc
and when it doesn't happen l am disillusioned

The weekly times is especially bad for this

I probably need to take more time to explain my frustrations more coherently and in greater detail with lots more substance

I do take note of forecasts in herald sun and rate them according to if they happened. ;lately
Try it yourself with the local paper forecast and experience that form of evaluation.

Maybe the public like me need educating to understand what the word forecast means, as it is not gospel

I read the primary industry forecast for farmers. How does a farmer interpret that . Slightly warmer, neural and above average?

Maybe the issue is the front of shop and not the back?

I don't mean to be offensive but l am at ground level. Your clientel . I am expressing a concern regarding interpreting the forecasting and as you said the old argument comes up time and time again. as to the humble reader of forecasts who make broad assumptions as to why we are not seeing what BOM sees in its exceptonal and extraordinary plefora of information

As to my comments regarding looking in to AWF forecasts , l am making a point that if people cannot get a confident handle on the information given then they will look elsewhere. or become anxious as the weather has been so strange and people are worried!!

The presentation and content of AWF forecasting .... applies to the AWF as well

I am sorry if l was offensive in venting my frustrations

Regards Crikey

[Meso]

Great reading this thread. Ken, do you work or did you work for the BoM in the past or something?

Anyway, totally agree on the heads up for thunderstorm outbreaks. Feb 3rd/4th was a good example where the main rain band/trough/whatever got a lot of media attention and was what was primarily talked about when warning the public of what could happen over those coming days. However, when we look back on it, it was actually the round of severe storms prior to what most were referring to as the "main course" coming through that did most of the damage (flooding). Many of us amateurs could see that a fairly big day or two of thunderstorms was on the cards a few days in advance. I assume the BoM did too, so I think more should have been made of this in the warnings and whatnot. A heads up of what they think is a high/medium/low risk days for thunderstorms what be great for us chasers too

Noticed you also mentioned 3D radar imagery programs, Ken. Is there anything like this available to the public, anywhere that you know of? I've heard a lot about GRLevel2 in the US, which sounds like it could be a similar thing. I really wish we had something like that here. But, I guess there just isn't the demand for it.

Maybe it would be more realistic to use terms like
chance of rain
likely showers
wind range south to SW
cloud: 20 to 80 % cover
showers 2 to 5mm
Rain 50 to 100mm
possible thunderstorm
Watch out for possible severe storms
temp: daytime high expected between 10 to 15 degrees

Crikey, you can get most of that info on WZ now. Check here - http://www.weatherzone.com.au/vic/north ... ry/bendigo" onclick="window.open(this.href);return false;

Has most of the things you mentioned and if you go in to the 48 hour forecast link it has %'s of cloud cover that are expected. Keep in mind, like most forecasts, it is also prone to some inaccuracies.