Sysmetrix weather sites
Sunday, November 2, 2003 by tommi_gi | Discussion: WinCustomize Talk
Does anyone have an alternate Metar site for the weather feature in Sysmetrix? The default site does not update every hour. Sometimes I have to wait 3-4 hours for a weather update to arrive.
Thank you
Thank you
tommi_gi
Reply #2 Sunday, November 2, 2003 1:35 PM
Reply #2 Sunday, November 2, 2003 1:35 PM
I can use the feature but the default site 
updates infrequently. Sometimes I'm looking at weather 4-5 hours old. I have another non-sysmetrix weather station that updates every 15 minutes but the default site will not work in the Sysmetrix program. In my case the NOAA site is the problem not the program.
updates infrequently. Sometimes I'm looking at weather 4-5 hours old. I have another non-sysmetrix weather station that updates every 15 minutes but the default site will not work in the Sysmetrix program. In my case the NOAA site is the problem not the program.
tommi_gi
Reply #3 Sunday, November 2, 2003 1:36 PM
Reply #3 Sunday, November 2, 2003 1:36 PM
I can use the feature but the default site; http://weather.noaa.gov/cgi-bin/mgetmetar.pl?cccc=[METARCODE]
updates infrequently. Sometimes I'm looking at weather 4-5 hours old. I have another non-sysmetrix weather station that updates every 15 minutes but the default site will not work in the Sysmetrix program. In my case the NOAA site is the problem not the program.
updates infrequently. Sometimes I'm looking at weather 4-5 hours old. I have another non-sysmetrix weather station that updates every 15 minutes but the default site will not work in the Sysmetrix program. In my case the NOAA site is the problem not the program.
Chickie
Reply #6 Sunday, November 2, 2003 2:18 PM
Me too.
Sometimes, I check Yahoo weather. 
Reply #6 Sunday, November 2, 2003 2:18 PM
| I just look putside to see the weather. |
Me too.
Sometimes, I check Yahoo weather. 
Darco
Reply #7 Sunday, November 2, 2003 11:21 PM
Reply #7 Sunday, November 2, 2003 11:21 PM
That is the site I use, and it updates whenever I click the weather button if the skin has it set up to, if not do you have the check in the configuration under Weather to check every hour. You can also right click on weather and choose Check Weather. It also tells you the last check and the last report and how old the report is. I just checked mine and it was 1 hour and 20 min.old, so I clicked on check weather on the bottom and it updated it.
Darco
Reply #8 Tuesday, November 4, 2003 4:22 PM
Reply #8 Tuesday, November 4, 2003 4:22 PM
Ok I see what you mean. I just checked mine again today and it was 5 hrs old and didn't update. That is a Government site isn't it? 
bakerstreet
Reply #9 Tuesday, November 4, 2003 4:58 PM
Reply #9 Tuesday, November 4, 2003 4:58 PM
This is an issue that really, really needs to be resolved. I posted on this over a year ago: ( https://www.wincustomize.com/msgboard.asp?BID=WC&id=56702 )
What the average person doesn't understand is that scripts that grab weather information from weather_dot_com or the NOAA site rarely ( if ever ) have permission to do so. Essentially what they are doing is parsing out a bit of HTML based on a request just like your browser sends. Even though you use only a couple of characters out of a page, most scripts parse them out from the whole thing.
This doesn't seem like much until you count how many people are using the script, and how often they have it set to update. If a thousand people use a script that polls the weather site every 5 minutes, that is 12,000 non-visiting hits an hour. I feel safe saying that a lot more people than that use Sysmetix alone. That doesn't count all the scripts for Samurize, CoolMon, etc, that update every 5 minutes or less as well.
Can you imagine how many hits they get from these scripts? Samurize was asked by several sites at not to carry scripts that polled them for news. The default polling was like 3 minutes and thousands of people were running the scripts. Hundreds of thousands of pointless hits a day were effecting their site performance. That is nutty traffic, especially when you consider they never really visit your sites, click your banners, etc.
I am still very dubious about any script that parses information from a commercial or governmental website and presents it as if it were raw data. It isn't, and the sites in question shouldn't bear the weight of these scripts unless they volunteer the service with a feed. If you do use them, it is really rude to have them update more often than the site itself updates.
[Message Edited]
What the average person doesn't understand is that scripts that grab weather information from weather_dot_com or the NOAA site rarely ( if ever ) have permission to do so. Essentially what they are doing is parsing out a bit of HTML based on a request just like your browser sends. Even though you use only a couple of characters out of a page, most scripts parse them out from the whole thing.
This doesn't seem like much until you count how many people are using the script, and how often they have it set to update. If a thousand people use a script that polls the weather site every 5 minutes, that is 12,000 non-visiting hits an hour. I feel safe saying that a lot more people than that use Sysmetix alone. That doesn't count all the scripts for Samurize, CoolMon, etc, that update every 5 minutes or less as well.
Can you imagine how many hits they get from these scripts? Samurize was asked by several sites at not to carry scripts that polled them for news. The default polling was like 3 minutes and thousands of people were running the scripts. Hundreds of thousands of pointless hits a day were effecting their site performance. That is nutty traffic, especially when you consider they never really visit your sites, click your banners, etc.
I am still very dubious about any script that parses information from a commercial or governmental website and presents it as if it were raw data. It isn't, and the sites in question shouldn't bear the weight of these scripts unless they volunteer the service with a feed. If you do use them, it is really rude to have them update more often than the site itself updates.
[Message Edited]
tommi_gi
Reply #10 Tuesday, November 4, 2003 7:32 PM
Reply #10 Tuesday, November 4, 2003 7:32 PM
BakerStreet,
I read your thread from 2002 and understand your premise. Here are some thoughts. First the METAR system is much like the time sync sites designed to synchronize system clocks with atomic time. The site is designed for thousands of hits per hour with a duration of 200 countries worldwide with many countries offering multiple dozens or even hundreds of city locations in either WMO index number or ICAO location indicator codes.
One way to help speed the system is to replace the [METARCODE] in the address string with the exact ICAO location indicator of the site you are monitoring. In my case I am in Los Angeles so I replace it with [KLAX]. When I am traveling I replace this with the site closest to my destination. This refined method reduces the cycle time from 1 second to .8 seconds (broadband) since I am not downloading the entire list and then asking Sysmetrix to pull the KLAX line out of the text file. In some weather programs you can pull the data for several sites along a flight path to gather a string of data for the planned route over land. But as I have studied this I realized that my inquiry connections always go through successfully (and instantly) but there is no updated data available. >> So I have surmised that they are bottlenecked in the collection end and perhaps distribution end and not the dissemination web site.<<
It is possible that the Sysmetrix program’s use is so extensive and the weather feature is on by default that the program is responsible for a bottleneck but I now doubt that is the case. I wonder if NOAA offers a subscription service in addition to the free service our tax dollars pay for and devote more resources to it than the free. This, I feel is also unlikely.
All the other METAR sites I have investigated route back to NOAA and thereby offer no benefit. Unlike atomic clock sites which seem to be numbered in the dozens or perhaps hundreds, there seems to be only one depository of the METAR data. I have learned that the automated weather sites report every hour to NOAA so it would seem they have the monopoly on the collection and distribution.
So if anyone finds an alternate site please post it here. Maybe we can spread out the load if we can find a couple new sources to use.
Lastly to those that suggest “opening a window” please disable the feature in Sysmetrix. It’s possible the mistake was made by not forcing this feature ‘off’ by default so only someone really interested would be pulling the data and thereby taxing the NOAA system. I agree that this info isn’t really meant for everyone.
Tommi
I read your thread from 2002 and understand your premise. Here are some thoughts. First the METAR system is much like the time sync sites designed to synchronize system clocks with atomic time. The site is designed for thousands of hits per hour with a duration of 200 countries worldwide with many countries offering multiple dozens or even hundreds of city locations in either WMO index number or ICAO location indicator codes.
One way to help speed the system is to replace the [METARCODE] in the address string with the exact ICAO location indicator of the site you are monitoring. In my case I am in Los Angeles so I replace it with [KLAX]. When I am traveling I replace this with the site closest to my destination. This refined method reduces the cycle time from 1 second to .8 seconds (broadband) since I am not downloading the entire list and then asking Sysmetrix to pull the KLAX line out of the text file. In some weather programs you can pull the data for several sites along a flight path to gather a string of data for the planned route over land. But as I have studied this I realized that my inquiry connections always go through successfully (and instantly) but there is no updated data available. >> So I have surmised that they are bottlenecked in the collection end and perhaps distribution end and not the dissemination web site.<<
It is possible that the Sysmetrix program’s use is so extensive and the weather feature is on by default that the program is responsible for a bottleneck but I now doubt that is the case. I wonder if NOAA offers a subscription service in addition to the free service our tax dollars pay for and devote more resources to it than the free. This, I feel is also unlikely.
All the other METAR sites I have investigated route back to NOAA and thereby offer no benefit. Unlike atomic clock sites which seem to be numbered in the dozens or perhaps hundreds, there seems to be only one depository of the METAR data. I have learned that the automated weather sites report every hour to NOAA so it would seem they have the monopoly on the collection and distribution.
So if anyone finds an alternate site please post it here. Maybe we can spread out the load if we can find a couple new sources to use.
Lastly to those that suggest “opening a window” please disable the feature in Sysmetrix. It’s possible the mistake was made by not forcing this feature ‘off’ by default so only someone really interested would be pulling the data and thereby taxing the NOAA system. I agree that this info isn’t really meant for everyone.
Tommi
bakerstreet
Reply #11 Tuesday, November 4, 2003 8:37 PM
Reply #11 Tuesday, November 4, 2003 8:37 PM
tommi_gi ,
If things have changed in the past year as to how these scripts work, forgive the oversight, I kinda gave up and moved on. When I was pondering it for my own work, I thought maybe that data collection could occur at the developer level, and then could be disseminated to the end user via a more efficient feed, like a database query.
For instance, a central server could spider the NOAA site periodically for the major metropolitan areas of every state. If you figure 3 to 5 queries per state, this would only be 150-250 queries for the US. Then the gathered data could be inserted into a database by area, efficiently, translated into data more readable by the scripts ( a single hash for current conditions and forecast, etc ) .
The actual polling of the weather source would take place only once per update period, per locale polled, instead of once for every end-user. Users would simply ping the developer's database to check for an update, and then get the bare minimum of what is needed, and only when there is new data. Much less traffic for weather sites, and much more efficient data for the end-user's software to manage. i.e. decoding a small string instead of parsing HTML.
Developers would essentially be mirroring the information, but in a much more useful and efficient form than HTML. Also, when the HTML format of the weather site changed, only the developer's spider would be effected, instead of breaking the end-users' scripts.
Again, this suggestion may be outdated, since it is based on the various solutions I worked on about a year ago. My apologies if so.
[Message Edited]
If things have changed in the past year as to how these scripts work, forgive the oversight, I kinda gave up and moved on. When I was pondering it for my own work, I thought maybe that data collection could occur at the developer level, and then could be disseminated to the end user via a more efficient feed, like a database query.
For instance, a central server could spider the NOAA site periodically for the major metropolitan areas of every state. If you figure 3 to 5 queries per state, this would only be 150-250 queries for the US. Then the gathered data could be inserted into a database by area, efficiently, translated into data more readable by the scripts ( a single hash for current conditions and forecast, etc ) .
The actual polling of the weather source would take place only once per update period, per locale polled, instead of once for every end-user. Users would simply ping the developer's database to check for an update, and then get the bare minimum of what is needed, and only when there is new data. Much less traffic for weather sites, and much more efficient data for the end-user's software to manage. i.e. decoding a small string instead of parsing HTML.
Developers would essentially be mirroring the information, but in a much more useful and efficient form than HTML. Also, when the HTML format of the weather site changed, only the developer's spider would be effected, instead of breaking the end-users' scripts.
Again, this suggestion may be outdated, since it is based on the various solutions I worked on about a year ago. My apologies if so.
[Message Edited]
tommi_gi
Reply #12 Thursday, November 6, 2003 8:04 PM
Reply #12 Thursday, November 6, 2003 8:04 PM
Today I decided to bare down on this and find out why I wasn't getting updated weather for >5 hours on the METAR sites near my house. I took about 1 hour and experimented with various METAR sites and I learned something interesting. Many sites from the USA are reporting updates every hour or less and many aren't. I live in LA. First I went to a popular flight data web site and entered my local ICAO info. All the sites had the same outdated info as was displayed through Sysmetrix. You can tell this by learning to read the time code and adjust from UTC time to your local time to see how old the data is OR you can use Sysmetrix as it will decode the time difference for you. But then I saw some "canned" METAR links for various popular airports so I clicked on a few and noticed the update times for some were outdated and some were updating?!?! So I went back to my state's listing and selected 10 ICAO codes from my general area (Southern California) and entered each one in Sysmetrix and hit the "Get Weather" button for each setting. 4 updated within 1/2 to 1 hour and 6 "outdated" back to the >5 hour info.
>>So I realized it must be the automated sending site for some METAR data is not refreshing regularly or is not sending to NOAA or NOAA is not processing the data from some sites in a timely manner<<
Therefore you will have to experiment with several local sites to find one that is working and use that one. I'm lucky that there are so many in the LA area to pick from. If you have only one in your area you might want to find a longitude and latitude map and get the closest one along the same 'long' and 'lat' track with similar geographic conditions. Otherwise perhaps you can contact NOAA and bitch at them to fix your local remote site problem. I suggest doing this anyway because our tax dollars pay for this stuff and if enough people complain they will fix the broken pieces and get these stations updating as they were intended which is at least 1 time per hour but as often as 4 times per hour.
Otherwise just open the window if that's all you need to know
)
And in case you are interested, go to this address to see a picture of an automated METAR weather station:
http://www.met.tamu.edu/class/METAR/metar.html
Tommi
>>So I realized it must be the automated sending site for some METAR data is not refreshing regularly or is not sending to NOAA or NOAA is not processing the data from some sites in a timely manner<<
Therefore you will have to experiment with several local sites to find one that is working and use that one. I'm lucky that there are so many in the LA area to pick from. If you have only one in your area you might want to find a longitude and latitude map and get the closest one along the same 'long' and 'lat' track with similar geographic conditions. Otherwise perhaps you can contact NOAA and bitch at them to fix your local remote site problem. I suggest doing this anyway because our tax dollars pay for this stuff and if enough people complain they will fix the broken pieces and get these stations updating as they were intended which is at least 1 time per hour but as often as 4 times per hour.
Otherwise just open the window if that's all you need to know
)And in case you are interested, go to this address to see a picture of an automated METAR weather station:
http://www.met.tamu.edu/class/METAR/metar.html
Tommi
tommi_gi
Reply #13 Saturday, November 15, 2003 10:26 PM
Reply #13 Saturday, November 15, 2003 10:26 PM
On November 14 the NOAA Metar weather site began updating again on an hourly basis. It would appear they had some problems which were resolved on Thursday and now it is working properly again. 
Tommi
Tommi
Please login to comment and/or vote for this skin.
Welcome Guest! Please take the time to register with us.
There are many great features available to you once you register, including:
- Richer content, access to many features that are disabled for guests like commenting on the forums and downloading skins.
- Access to a great community, with a massive database of many, many areas of interest.
- Access to contests & subscription offers like exclusive emails.
- It's simple, and FREE!
Reply #1 Sunday, November 2, 2003 1:27 PM