Automatically Export & Sync Shopify Data to Google BigQuery
Some of the Shopify data sets Flatly offers include metafields and metaobjects automatically blended in with each parent data set (example: Orders-Last X Days+MetaFields). Since these metas can contain nested data, Flatly outputs these data sets in a left-joined format. The parent to each meta has its fields repeated in each row, whereas each meta is unique to each row. The labels from each meta are presented as column headings, to make the data intelligible for human users.
A non-left-joined option (high column count) is available as well. First, experiment with the existing Flatly Shopify data sets and check if your business intelligence platform can properly relate repeated parents to unique metas without any additional setup. Reach out to Flatly Support for more details.
Flatly's access to Plaid and Plaid's downstream connected financial institutions is read-only access to text. It is limited to account data (words, numbers, dates). It does not include any money-movement, transfer capabilities or account holder profiles/identities.
Plaid alone interfaces directly with financial institutions, caches data from those institutions on Plaid infrastructure, and then makes the appropriate scoped subset of that data available to partners like Flatly using secure connections called client libraries.
Flatly leverages intelligent schema generation that adapts to BigQuery's auto-detection protocols, handling data type inference and column ordering automatically so you never have to manually map fields.
Data hygiene is built directly into the ingestion process. Before a single row is written to BigQuery, Flatly validates your data against a generated reference schema. It proactively detects and strips out "schema violators" —inconsistent data types or malformed records that would typically cause a load job to crash. Flatly logs these exclusions in a detailed metadata file, ensuring your pipeline remains resilient and continuous while giving you full transparency into data quality issues.
Designed for scale and flexibility, the engine processes data in memory-optimized chunks to handle massive datasets efficiently. It offers powerful pre-load controls, allowing you to filter rows based on custom logic (e.g., date ranges, specific values) and allowlist specific keys. This ensures that you only sync high-value data, optimizing your BigQuery storage costs and query performance. Everything is secured via Service Account authentication, providing a stable, production-grade connection for your most critical analytics dashboards.
Connect your data with our turnkey data integration solution so you can focus on running your business.
