HomeBlogContact Sales
Changelog
Back to All

May-2021 Release Notes

over 3 years ago by [email protected]

May 2021 is nothing short of a landmark release for SafeGraph🍾 . Catch the highlights below! (2021-04-29/1619683326 shipped 2021-05-05).

Highlights

  • Great Britain Core and Geometry is live! 🇬🇧
  • Canada Weekly Patterns is live! 🇨🇦
  • Humongous brand additions across all geographies 🎉
  • Peep our revamped Summary Stats page to track progress on key product metrics 📈

Table of Contents:

Core Places and Brands

Enhancements - Core Places and Brands

  • Fresh off a multi-month sprint, we are thrilled to announce our MVP release of SafeGraph Places across the pond! 🇬🇧 Great Britain Places features 1.3MM+ POIs, 500+ brands, geometry, and all the same spatial hierarchy and polygon metadata concepts as applied in the US and Canada 💯 . Contact your CSM to learn more or add to your subscription.

  • Last month, SG Places had 6,986,485 points-of-interest (including closed POIs). This month, SG Places has 8,368,418 points-of-interest (net 1,381,933 places). These are +6,283 US Places 🇺🇸 , +1,321 CA places 🇨🇦, and +1,312,112 GB places 🇬🇧 .

  • We've added 648 brands ‼️ (no, that's not a typo - view the full list here). Below are some highlights:

    • +106 Full-Service Restaurant brands (722511) 🍴
    • +99 Limited-Service Restaurant brands (722513) 🍔
    • +507 brands with UK coverage
    • +250 brands with CA coverage
    • +179 brands with US coverage

Brand Openings and Closings

  • We rely on POI metadata to track store openings and closings, and we are especially interested in understanding open/close dates for branded POIs. It can take more than a month to infer open/close dates, so we report brand open/close metrics on a one month delay.

Bug Fixes and Known Issues - Core Places and Brands

  • We discovered a few brand count fluctuations as a result of updated sourcing and metadata bugs. These corrections resulted in significant changes in the total number of POIs for each affected brand, but the new count is correct. For transparency, we'd like to list some of these corrections as examples in no particular order:
    • Comerica (SG_BRAND_5b6664e29e3855655d53f5ad83637421). Net POI count change: US: -85 CA: 0. Bug: Previously included ATM-only locations.
    • Floors to Go (SG_BRAND_17b1cf0ea1fe75f3dd28149bb9d6af49). Net POI count change: US: -267 CA: 0. Bug: Previously included duplicated with Abbey Carpet and Floor (SG_BRAND_faa9096ceee3f8943ec22bde486bd15f).
    • BBVA Compass Bancshares (SG_BRAND_171715760808fad662a86e8c2028b0a0). Net POI count change: US: -650 CA: 0. Bug: Previously included ATM-only locations.
    • West Coast Hooters (SG_BRAND_e5c3d8f947863b5e0e7dcf424e8fbe0c). Net POI count change: US: -7 CA: 0. Bug: All were already included in Hooters brand (SG_BRAND_109461c56973d3e435e5476731734b87).
    • United Bank (SG_BRAND_3edd0a114c0d2e78145a9fb327e89ff2). Net POI count change: US: -60 CA: 0. Bug: Previously included ATM-only locations.

Enhancements - Categories

We continue to focus on expanding our coverage across industrial places 🏭 📦 . This month, we added:

  • +1,257 data centers (518210)
  • +849 "last mile" distribution centers (493110) for Amazon Distribution (SG_BRAND_fc2573e1b20d6dd1) 🚚

Category Fill Rate -- We monitor category fill rate with 2 metrics: (1) category fill rate across the entire dataset, and (2) category fill rate for branded POI. We want both of these numbers to be 100%.
(1) All POI category fill rate. Last month 99.2%. This month 99.4%. 📈
(2) Branded POI category fill rate. Last month 100%. This month 100% 💯

Drops ⬇️

We ingest data from many sources, and due to source changes and processing changes, Placekeys churn over time. In this release, we dropped 271,479 Placekeys (72,645 branded and 198,834 non-branded). To keep track of the status, predecessors, and latest successor of each Placekey, hit the Lineage API for free!

Major reasons for drops:

  • As a result of improved address matching, ~208k Placekeys changed from a zzy address encoding (invalid Placekey address) to a non-zzy address encoding (valid Placekey) 👏
  • ~27k dropped as result of improved deduplication 👯
  • ~1k dropped as a result of bug fixes for branded POIs 🐛
  • ~18k dropped due to changes to the Where part
  • Read more about the structure of Placekeys here

Geometry

Enhancements - Geometry

  • While OWNED polygons are preferred, it does not mean that SHARED polygons are inherently bad. It only means that the exact shape of each POI within the polygon is not discernible, but the general location can be identified by the centroid (latitude & longitude). 🎯

  • When enclosed = FALSE, it indicates that there are reasonable means to derive a unique polygon for the POI (even when parent_placekey is not null), and we strive for 100% of branded, non-enclosed POIs to have polygon_class = "OWNED_POLYGON."

  • Last month, the percent OWNED polygons for branded, non-enclosed POIs was 76.6%

  • This month, the percent OWNED polygons for branded, non-enclosed POIs is 73.0% 📉 -- this number decreased due to the MVP GB release but will steadily increase as preferred polygons are incrementally sourced in Great Britain.

Bug Fixes and Known Issues - Geometry

Patterns

Enhancements - Patterns

  • Weekly Patterns is now available in Canada! 🇨🇦 Contact your CSM to learn more or add to your subscription.

    • Please note: As a side effect of adding Canada Weekly Patterns, both Weekly and Monthly Patterns for U.S. customers will have Canada Dissemination Areas in the visitor_home_cbgs and visitor_daytime_cbgs column (whereas before it only included U.S. census block groups or states). The format of the dissemination areas will be like this: "CA:1209010302". The format of the U.S. census block groups will remain as they have always been. This may cause an issue for those who are ingesting and validating the column to only have the CBG format. On the plus side, it will provide insights into visitors originating from Canada.
    • Customers will also have new rows for Canadian dissemination areas in home_panel_summary and Canadian provinces in visit_panel_summary and normalization_stats, as well as a new column iso_country_code added as the rightmost column of each file.

  • In last month's delivery, SG Monthly Patterns had 4,456,989 points-of-interest (US only). This month, SG Monthly Patterns has 4,472,356 points-of-interest (net +15,367).

  • Last month, SG Monthly Patterns had 966,668,399 visits from 31,035,273 visitors (US only). This month, SG Monthly Patterns has 971,170,009 visits from 32,877,967 visitors (delta +4,501,610 visits, +1,842,694 visitors).

  • Neighborhood Patterns, which provides more generalized foot traffic flows across census block groups, saw 2,506,425,606
     raw stops (+419,694,819 from last month) from 412,688,578 raw devices (+14,274,854 from last month).

Interested in global POI coverage? Reach out to your customer success manager to learn more about how we're thinking about growing coverage internationally. 🌎 

**In case you missed it,** check out [last month's release notes](https://docs.safegraph.com/changelog/april-2021-release-notes). 📝

**Calculating Diffs**
Curious to find the specific records that were either **added, deleted, or saw an attribute change** from one release to the next? Visit "Calculating Diffs" in our [Data Science Resources](https://docs.safegraph.com/docs/data-science-resources#section-calculating-diffs) to get started. 

**Fill Rates**
See the [Summary Statistics](https://docs.safegraph.com/docs/places-summary-statistics) page for all Core and Geometry column fill rates as well as a breakdown of POI count by `naics_code`.

**Explore**
Browse SafeGraph Core & Geometry data at your own pace [in these webmaps.](https://storymaps.arcgis.com/stories/8e5e066486f94f0ea698e507d46987f7)

**Also check out these new ways to get SafeGraph data: **
* Need data on the fly? [Try our Places API](https://shop.safegraph.com/api)!  
* Need some extra data or other SafeGraph products? Check out the [SafeGraph Data Bar.](https://shop.safegraph.com/) 
* Heavy AWS User?  Check out our [listings in the AWS Data Exchange](https://aws.amazon.com/marketplace/search/results?filters=vendor_id&vendor_id=7d5ff8ca-105f-4856-9d99-5f2f1d83223c).
* Snowflake user? Check out our page on the [Snowflake Data Exchange](https://www.snowflake.com/datasets/safegraph/) :snowflake: 
* Or just drop us a line! Your data needs are our data delights!