HomeBlogContact Sales
Changelog
Back to All

June-2021 Release Notes

over 3 years ago by [email protected]

Welcome to the June 2021 release notes. 📝 Catch the highlights below! (2021-05-28/1622229555 shipped 2021-06-04).

Highlights

  • Total POIs with Patterns, raw_visit_counts, and raw_visitor_counts approaching pre-pandemic levels 📈
  • Record high precise polygons 🔥

Table of Contents:

Core Places and Brands

Enhancements - Core Places and Brands

  • Last month, SG Places had 8,368,418 points-of-interest (including closed POIs). This month, SG Places has 8,413,852 points-of-interest (net 45,434 places). These are +44,402 US Places 🇺🇸 , +1,374 CA places 🇨🇦, and -342 GB places 🇬🇧 .
  • We've added 91 brands ‼️ (+75 🇺🇸, +22 🇨🇦, +27 🇬🇧) view the full list here. Below are some highlights:
    • Co-op Food Store (SG_BRAND_4ca73eedcd36fd97) with 3,715 GB Places
    • One Stop (SG_BRAND_109d4852764fbd37) with 803 GB Places
    • Health Mart (SG_BRAND_102c570f10f65fef82fad50e8839dbb6) with 717 US Places
    • +25 Limited-Service Restaurants brands (722513) 🍔
    • +14 Convenience Stores brands(445120) 🏪
    • +13 Full-Service Restaurants brands (722511) 🍴

Brand Openings and Closings

  • We rely on POI metadata to track store openings and closings, and we are especially interested in understanding open/close dates for branded POIs. It can take more than a month to infer open/close dates, so we report brand open/close metrics on a one month delay.

Bug Fixes and Known Issues - Core Places and Brands

  • The region column in Great Britain = UK county, and we migrated to a higher quality UK county source. As a result, ~457k GB region values changed, and we expect these to remain stable going forward.

Enhancements - Categories

This month, key changes include:

  • +4,636 supermarkets (445110) 🍞
  • +767 pharmacies (446110) 💊
  • 9,923 POI changed from General Warehousing and Storage (493110) to Lessors of Miniwarehouses and Self-Storage Units (531130). 📦
  • 2,076 POI changed from Full Service Restaurants (722511) to Restaurants and Other Eating Places (722515). 🍦
  • 1,850 POI changed from Elementary and Secondary Schools (611110) to Child Day Care Services (624410). 🚸
  • 1,836 POI changed from Child Day Care Services (624410) to Elementary and Secondary Schools (611110) 🏫

Category Fill Rate -- We monitor category fill rate with 2 metrics: (1) category fill rate across the entire dataset, and (2) category fill rate for branded POI. We want both of these numbers to be 100%.
(1) All POI category fill rate. Last month 99.4%. This month 99.4%.
(2) Branded POI category fill rate. Last month 100%. This month 100% 💯

Drops ⬇️

We ingest data from many sources, and due to source changes and processing changes, Placekeys churn over time. In this release, we dropped 167,708 Placekeys (46,243 branded and 121,465 non-branded). To keep track of the status, predecessors, and latest successor of each Placekey, hit the Lineage API for free!

Major reasons for drops:

  • As a result of improved address matching, ~32k Placekeys changed from a zzy address encoding (invalid Placekey address) to a non-zzy address encoding (valid Placekey) 👏
  • ~21k dropped as result of improved deduplication 👯
  • ~23k dropped due to changes to the Where part
  • Read more about the structure of Placekeys here

Geometry

Enhancements - Geometry

  • While OWNED polygons are preferred, it does not mean that SHARED polygons are inherently bad. It only means that the exact shape of each POI within the polygon is not discernible, but the general location can be identified by the centroid (latitude & longitude). 🎯

  • When enclosed = FALSE, it indicates that there are reasonable means to derive a unique polygon for the POI (even when parent_placekey is not null), and we strive for 100% of branded, non-enclosed POIs to have polygon_class = "OWNED_POLYGON."

  • Last month, the percent OWNED polygons for branded, non-enclosed POIs was 73.0%

  • This month, the percent OWNED polygons for branded, non-enclosed POIs is 74.5% 📈 -- this number will steadily increase as preferred polygons are incrementally sourced in Great Britain.

Bug Fixes and Known Issues - Geometry

  • Centroid-Radius Polygons -- As discussed in March 2019 release notes. We internally track centroid-radius polygons vs precise polygons and strive for 100% precise polygons. You can measure this yourself using the is_synthetic column.
  • Last release, the percent of precise polygons was 95.9%
  • This release, the percent precise polygons increased to 96.3% 📈, as a result of ongoing improvements to geometry logic for parent-child POI and reducing overlapping polygons.

Patterns

Enhancements - Patterns

  • In last month's delivery, SG Patterns had 4,472,356 points-of-interest (US only). This month, SG Patterns has 4,511,670 points-of-interest (US only) (net 39,314).

  • Last month, SG Patterns had 971,169,984 visits from 32,877,968 visitors. This month, SG Patterns has 1,031,959,744 visits from 35,337,908 visitors (delta 60,789,708 visits, 2,459,940 visitors).

  • In our Neighborhood Patterns product, where you can see more generalized foot traffic flows, we have:

    • 2,205,446,144 raw stops (-300,979,488 from last month)
    • 454,091,296 raw devices (41,402,720 from last month)

  • Weekly Patterns is now available in Canada! 🇨🇦 Contact your CSM to learn more or add to your subscription.

    • Please note: As a side effect of adding Canada Weekly Patterns, both Weekly and Monthly Patterns for U.S. customers will have Canada Dissemination Areas in the visitor_home_cbgs and visitor_daytime_cbgs column (whereas before it only included U.S. census block groups or states). The format of the dissemination areas will be like this: "CA:1209010302". The format of the U.S. census block groups will remain as they have always been. This may cause an issue for those who are ingesting and validating the column to only have the CBG format. On the plus side, it will provide insights into visitors originating from Canada.
    • Customers will also have new rows for Canadian dissemination areas in home_panel_summary and Canadian provinces in visit_panel_summary and normalization_stats, as well as a new column iso_country_code added as the rightmost column of each file.
Interested in global POI coverage? Reach out to your customer success manager to learn more about how we're thinking about growing coverage internationally. 🌎 

**In case you missed it,** check out [last month's release notes](https://docs.safegraph.com/changelog/may-2021-release-notes). 📝

**Calculating Diffs**
Curious to find the specific records that were either **added, deleted, or saw an attribute change** from one release to the next? Visit "Calculating Diffs" in our [Data Science Resources](https://docs.safegraph.com/docs/data-science-resources#section-calculating-diffs) to get started. 

**Fill Rates**
See the [Summary Statistics](https://docs.safegraph.com/docs/places-summary-statistics) page for all Core and Geometry column fill rates as well as a breakdown of POI count by `naics_code`.

**Explore**
Browse SafeGraph Core & Geometry data at your own pace [in these webmaps.](https://storymaps.arcgis.com/stories/8e5e066486f94f0ea698e507d46987f7)

**Also check out these new ways to get SafeGraph data: **
* Need data on the fly? [Try our Places API](https://shop.safegraph.com/api)!  
* Need some extra data or other SafeGraph products? Check out the [SafeGraph Data Bar.](https://shop.safegraph.com/) 
* Heavy AWS User?  Check out our [listings in the AWS Data Exchange](https://aws.amazon.com/marketplace/search/results?filters=vendor_id&vendor_id=7d5ff8ca-105f-4856-9d99-5f2f1d83223c).
* Snowflake user? Check out our page on the [Snowflake Data Exchange](https://www.snowflake.com/datasets/safegraph/) :snowflake: 
* Or just drop us a line! Your data needs are our data delights!