FEMA / SAME QUERY FAILS - SSL Issues and Excessive queries to FEMA #63

New Issue

2026-10-05T01:18:57+02:00

adam commented

2026-10-05 01:18:57 +02:00

FEMA QUERY FAILS

OS: Ubuntu 20.04

====

SSL CERT ISSUES - FEMA/IPAWS
In our case we kept seeing rejections of obtaining records from FEMA / SAME url. This was due to some type of SSL issues.
We updated our SSL on server - But still encountered SSL failures.

BAD FIX:
locationdata.py - Near line: 500 - Inside for loop for entry.

Find requests.get and add verify=False as argument for requests.get. ** NOT A GOOD SOLUTION - BUT WORKS**

The above will introduce warnings in the logs and console about the SSL. To bypass warnings from show in logs/console - Add the below to top of locationdata.py

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

====

EXCESSIVE FEMA QUERIES TO FEMA/IPAWS

Once the above was fixed we noticed that the FEMA/SAME query accessed a feed from FEMA that would return a link for every possible alert across the country.

The code would then follow EACH link being retuned from FEMA and would THEN check the SAME codes from the FEMA alert to see if it matched our config.ini SAME codes. This causes a SLAM of queries against the FEMA servers and may not be needed with a minor code change.

Within the initial FEED query that happens every 20 minutes from meshing around - That data returns a StateFIP code for where the alert is. By simply adding a little code and a StateFIPS list to our config.ini - We can pre-filter out what URLS we will access.

In other words... If we are in Ohio - We will not follow alert links for california, texas, utah, etc. - Thereby reducing the load on FEMA servers and our OS.

FIX:
We added a validation to first check the StateFIP code to see if it matches our state (or list) BEFORE following each link. This reduces the load on the FEMA servers and also reduces load on our side. Below is our fix:

locationdata.py

# extract alerts from main feed
for entry in alertxml.getElementsByTagName("entry"):
    link = entry.getElementsByTagName("link")[0].getAttribute("href")


    ## state FIPS
    ## This logic is being added to reduce load on FEMA server.   If the initial feed list doesn't match
    ## our state - Then there is no point in following each link.

    stateFips = None
    for cat in entry.getElementsByTagName("category"):
        if cat.getAttribute("label") == "statefips":
            stateFips = cat.getAttribute("term")
            break

    if stateFips is None:
        # no stateFIPS found — skip
        continue

    # check if it matches your list
    if stateFips not in configStateFIPSList:
        logger.debug(f"Skipping FEMA record link {link} with stateFIPS code of: {stateFips} because it doesn't match our StateFIPSList {configStateFIPSList}")
        continue  # skip to next entry

    ## END state FIPS

settings.py
configStateFIPSList = config['location'].get('myFIPSList', '').split(',') # default empty

config.ini

comma sep list of stateFIPS codes - This reduces load on FEMA server by not pulling alerts that are not in our state.

https://en.wikipedia.org/wiki/Federal_Information_Processing_Standard_state_code

myFIPSList = 39

====
OTHER NOTES
Since we are a public safety org working on various community projects - We had a conversation with the development team at FEMA/IPAWS.

Some notes they provided us:
1 - Please... No excessive queries to their servers at less than 2 minute intervals. Excessive queries from a single IP address could result in you being blocked.

Meshing-around notes: The default query time to hit FEMA is every 20 minutes to get the feed. But the bug above causes a "rapid fire" of every alert throughout the country. Slamming their servers. Where this could be 1 to 500 queries back to back depending on the number of alerts throughout the country.

2 - There is no longer a IPAWS PIN required for read only access.

Meshing-around notes: Since the meshing-around code adds ?pin=#### to the URL - We were concerned that the URL may break if no pin was specified. We confirmed that it DOES work with ?pin= at the end of the URL. - But a good fix to code would be NOT to put that there unless an IPAWS pin was specified within the config.ini

CLOSING...
The above fixes have only been through initial testing and we have not seen an alert within our state to make sure everything is fully functional. Please double check the code before including in repo.

Keep mesh alive! :)

FEMA QUERY FAILS OS: Ubuntu 20.04 ==== SSL CERT ISSUES - FEMA/IPAWS In our case we kept seeing rejections of obtaining records from FEMA / SAME url. This was due to some type of SSL issues. We updated our SSL on server - But still encountered SSL failures. BAD FIX: locationdata.py - Near line: 500 - Inside for loop for entry. Find requests.get and add verify=False as argument for requests.get. ** NOT A GOOD SOLUTION - BUT WORKS** The above will introduce warnings in the logs and console about the SSL. To bypass warnings from show in logs/console - Add the below to top of locationdata.py import urllib3 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) ==== EXCESSIVE FEMA QUERIES TO FEMA/IPAWS Once the above was fixed we noticed that the FEMA/SAME query accessed a feed from FEMA that would return a link for every possible alert across the country. The code would then follow EACH link being retuned from FEMA and would THEN check the SAME codes from the FEMA alert to see if it matched our config.ini SAME codes. This causes a SLAM of queries against the FEMA servers and may not be needed with a minor code change. Within the initial FEED query that happens every 20 minutes from meshing around - That data returns a StateFIP code for where the alert is. By simply adding a little code and a StateFIPS list to our config.ini - We can pre-filter out what URLS we will access. In other words... If we are in Ohio - We will not follow alert links for california, texas, utah, etc. - Thereby reducing the load on FEMA servers and our OS. FIX: We added a validation to first check the StateFIP code to see if it matches our state (or list) BEFORE following each link. This reduces the load on the FEMA servers and also reduces load on our side. Below is our fix: locationdata.py # extract alerts from main feed for entry in alertxml.getElementsByTagName("entry"): link = entry.getElementsByTagName("link")[0].getAttribute("href") ## state FIPS ## This logic is being added to reduce load on FEMA server. If the initial feed list doesn't match ## our state - Then there is no point in following each link. stateFips = None for cat in entry.getElementsByTagName("category"): if cat.getAttribute("label") == "statefips": stateFips = cat.getAttribute("term") break if stateFips is None: # no stateFIPS found — skip continue # check if it matches your list if stateFips not in configStateFIPSList: logger.debug(f"Skipping FEMA record link {link} with stateFIPS code of: {stateFips} because it doesn't match our StateFIPSList {configStateFIPSList}") continue # skip to next entry ## END state FIPS settings.py configStateFIPSList = config['location'].get('myFIPSList', '').split(',') # default empty config.ini # comma sep list of stateFIPS codes - This reduces load on FEMA server by not pulling alerts that are not in our state. # https://en.wikipedia.org/wiki/Federal_Information_Processing_Standard_state_code myFIPSList = 39 ==== OTHER NOTES Since we are a public safety org working on various community projects - We had a conversation with the development team at FEMA/IPAWS. Some notes they provided us: 1 - Please... No excessive queries to their servers at less than 2 minute intervals. Excessive queries from a single IP address could result in you being blocked. Meshing-around notes: The default query time to hit FEMA is every 20 minutes to get the feed. But the bug above causes a "rapid fire" of every alert throughout the country. Slamming their servers. Where this could be 1 to 500 queries back to back depending on the number of alerts throughout the country. 2 - There is no longer a IPAWS PIN required for read only access. Meshing-around notes: Since the meshing-around code adds ?pin=#### to the URL - We were concerned that the URL may break if no pin was specified. We confirmed that it DOES work with ?pin= at the end of the URL. - But a good fix to code would be NOT to put that there unless an IPAWS pin was specified within the config.ini CLOSING... The above fixes have only been through initial testing and we have not seen an alert within our state to make sure everything is fully functional. Please double check the code before including in repo. Keep mesh alive! :)

adam closed this issue

2026-10-05 01:18:57 +02:00

adam commented

2026-10-05 01:18:57 +02:00

switched the event pulling up thanks

switched the event pulling up thanks

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: adam/meshing-around#63