Delivery File Structure

❗️

Legacy Product

This page references SafeGraph Patterns, Weekly Patterns, and/or Neighborhood Patterns, legacy products that will no longer be available at the start of 2023. If you are interested in foot traffic data, please contact us and we can refer you to a mobility data partner.

Places File Delivery

Once we receive the name of your configured S3 bucket, SafeGraph will begin to deliver data once per month formatted per the SafeGraph Places schema. Please note we may push updates more frequently on occasion in the case of an urgent bug fix.

• If you are an enterprise customer, SafeGraph products are delivered (together) on the ~ 7th of the month.
  • Up to 5 file paths will be delivered with the following structure: s3://customer-bucket/customer-prefix/{{sg-file-name}}/yyyy/mm/dd/hh/*.csv.gz. {{sg-file-name}} is one of the following:
    • core_poi, geometry, patterns or some combination like core_poi-geometry or core_poi-patterns or core_poi-geometry-patterns (depending on your subscription) includes all of the following to which you are subscribed: Places, Geometry, Patterns.
    • Weekly Patterns deliveries have a separate partition to distinguish from monthly deliveries and have the following structure: s3://customer-bucket/customer-prefix/weekly/{{sg-file-name}}/yyyy/mm/dd/hh/*.csv.gz.
    • brand_info (if subscribed to Places)
    • home_panel_summary (if subscribed to Patterns)
    • visits_panel_summary (if subscribed to Patterns)
    • normalization_stats (if subscribed to Patterns)

The files are sent as gzipped CSVs. Given that our data deliveries contain a lot of data we break up the delivery into multiple small files. Let us know if this doesn't make the process easier on your side!

See also Column Ordering.

Delivery Frequency

• Customers who license Places or Geometry get monthly deliveries
• Customers who license Weekly Patterns (+core/geo optionally) get weekly deliveries and no monthly deliveries
• Customers who license Monthly Patterns (+core/geo optionally) get monthly deliveries and no weekly deliveries
• Customers who license both Weekly and Monthly Patterns (+Places/Geometry optionally) get monthly and weekly deliveries

Historical Patterns Delivery

From time to time SafeGraph will provide you with historical Patterns data. You may be a new customer receiving historical data, or an existing customer receiving historical backfill of the latest version. Historical/Backfill deliveries are handled specially by the SafeGraph team and delivered in a specific format as follows:

1. s3://customer-bucket/customer-prefix/{{products_ordered}}_backfill/del_yyyy/del_mm/del_dd/del_hh/data_yyyy/data_mm/*
2. s3://customer-bucket/customer-prefix/home_panel_summary_backfill/del_yyyy/del_mm/del_dd/del_hh/data_yyyy/data_mm/*
3. s3://customer-bucket/customer-prefix/vists_panel_summary_backfill/del_yyyy/del_mm/del_dd/del_hh/data_yyyy/data_mm/*
4. s3://customer-bucket/customer-prefix/normalization_stats_backfill//del_yyyy/del_mm/del_dd/del_hh/data_yyyy/data_mm/*

Where

• del_yyyy/del_mm/del_dd/del_hh is the datetime the backfill was delivered (hour is start hour)
• data_yyyy/data_mm represents year and month being summarized for this set of patterns data.
• {{products_ordered}} will be patterns or core_poi-patterns or core_poi-geometry-patterns, depending on which data products are being delivered.
• Note that the data_* fields will also align with the columns date_range_start and date_range_end contained in each file of the Patterns data itself.
• This delivery format was first introduced as part of the October 2019 release.