Stream GA4 to BigQuery from sGTM: Bypass the 1M Export Limit

If your GA4 BigQuery export stopped one morning with no warning, that is not a bug: it is the ceiling. The native daily export suspends once you pass a million events per day, and Google does not backfill what you lost. It is gone for good. The good news: since Google opened the BigQuery API inside the server container, you can bypass that limit by writing hits straight from sGTM to BigQuery, streaming, no ceiling. Here is how, what it costs, and, more importantly, when it is not worth the trouble.

The two GA4 BigQuery export limits nobody plans for

Everyone knows the native GA4 BigQuery export. It is free, it takes two clicks in the admin, and it is perfectly fine for most sites. The catch is that it has two limits you only discover the hard way.

The one-million ceiling. The daily export is capped at 1 million events per day. Above that, Google does not queue the extra events for later: it simply suspends the export for that day. Your data for the day is lost, with no recovery. For a high-traffic site or an e-commerce store during a sale, that is a hole in your history that nothing fills.

The delay. Even under the ceiling, the daily export lands 24 to 48 hours late, and Google backfills late events for up to three days afterward. In practice: yesterday’s table (events_YYYYMMDD) is not reliable if you query it too early. Your numbers will still move. Build an intraday dashboard on that, and you are telling stories.

I cover the native export and how to work with it in the GA4 BigQuery export guide: this article assumes you know it and focuses on the difference.

Three ways to feed BigQuery: the comparison

Before you build anything, put the options on the table. There are three, and the third is the one few people use yet.

The GA4 streaming export lifts the million ceiling and cuts the delay to about fifteen minutes, at roughly $0.05 per GB streamed. It is often the sufficient answer, and honestly, if your only problem is the ceiling, start there. Writing directly from sGTM becomes worth it when you want something GA4 cannot do: your own schema, server-side enrichment, or freshness that even the GA4 streaming export does not guarantee.

Writing from sGTM to BigQuery: the implementation

Now that the server container sandbox exposes the BigQuery API, a server-side tag can insert rows into a BigQuery table through BigQuery.insert. The function takes the connection details (project, dataset, table) and an array of rows, and returns a promise that resolves on success or rejects with a per-row list of errors. You need a working server container first: if you do not have one, start with the server-side GTM migration guide.

1. The service account and IAM role. Create a dedicated service account and grant it the BigQuery Data Editor role, scoped to the dataset, not the whole project. Least privilege applies here too: this account only needs to write to one table. If your container runs on Cloud Run with an attached service account, the API can authenticate automatically, no key to carry around.

2. The table schema. This is trap number one, so let me say it plainly: the schema you define is not the native GA4 export schema. No repeated event_params RECORD, no nested structure inherited from Google. You start from a blank slate. A minimal schema looks like this:

CREATE TABLE `project.dataset.events_direct` (
  event_name    STRING,
  event_ts      TIMESTAMP,
  client_id     STRING,
  user_id       STRING,
  page_location STRING,
  value         NUMERIC,
  currency      STRING,
  consent_state STRING,
  params        JSON
);

3. The server-side tag. In a custom tag (or via the google/sgtm-ga4-to-bigquery repo template), you read the event data, build the row, and call the insert:

const BigQuery = require('BigQuery');
const getAllEventData = require('getAllEventData');

const event = getAllEventData();
const connection = {
  projectId: 'project',
  datasetId: 'dataset',
  tableId: 'events_direct'
};
const rows = [{
  event_name: event.event_name,
  event_ts: (new Date()).toISOString(),
  client_id: event.client_id,
  value: event['x-ga-mp1-ev'] || 0,
  params: JSON.stringify(event)
}];

BigQuery.insert(connection, rows, {ignoreUnknownValues: true})
  .then(data.gtmOnSuccess, data.gtmOnFailure);

4. The Google repo, use it with eyes open. Google publishes a ready-made template (google/sgtm-ga4-to-bigquery) that handles batched events, Consent Mode v2, and Google Ads attribution. It saves you a lot of time. But the README says it in black and white: this is not a Google-supported product. No SLA, no official support. You adopt it, you maintain it. That is a perfectly reasonable choice, as long as you own it and do not drop it onto a critical pipeline unmonitored.

What direct writes unlock that the export never will

If it were only about dodging the ceiling, the GA4 streaming export would be enough. The real value of direct writes is elsewhere.

Enrichment before insert. At the server level, you control the data before it touches BigQuery. You can attach a corrected order value, inject a CRM segment, resolve geo, clean a broken parameter, all before the write. The native export takes whatever GA4 received, full stop. To go further on the server-to-CRM loop, see the GA4 Measurement Protocol on the server side.

Ad-blocker immunity. The hit leaves from the server, not the browser. A blocker that would have killed the client request can do nothing against a server-side insert. You capture events the native export, which depends on the client tag, never saw.

Freshness. Near real-time, without the fifteen minutes of GA4 streaming or the 24 to 48 hours of the daily export. For an operational dashboard that has to reflect the moment, that changes everything.

The traps to know before you switch

I am not going to sell you a dream. This approach has a cost, both literally and figuratively.

The schema is yours, so your existing queries break. Every query tuned to the GA4 export schema (the ones in the essential GA4 BigQuery queries guide, for instance) target event_params, user_properties, Google’s nested house structure. Against your direct table, they error out. You rewrite them, or you reproduce the GA4 schema exactly, which is tedious.

The streaming insert cost. The BigQuery.insert API goes through the legacy streaming insert, billed at around $0.05 per GB inserted, on top of storage and the Cloud Run that runs the container. It is cheap per GB, but at high volume it adds up. Do the full math, streaming insert plus server infrastructure, using what server-side GTM really costs.

Consent, always. Writing to a warehouse excuses nothing. If the user has not consented, you do not write their personal data, or you write it anonymized. Consent Mode v2 must be honored on the server side exactly as on the client. A direct pipeline that ignores consent is a compliance failure, not an optimization.

Decision tree: who should switch, who should stay

Here is how I call it on client work.

Stay on the native export if you are under a million events per day and J-1 freshness is enough. It is free, robust, the schema is standard, and the whole community knows how to query it. Do not add complexity for its own sake.

Move to the GA4 streaming export if you pass the million or want fifteen-minute freshness, but the GA4 schema suits you and you do not need to enrich the data. It is the best effort-to-result ratio for most overflow cases.

Write directly from sGTM if you stack at least two of these needs: massive volume, strict near real-time, server-side enrichment before insert, ad-blocker immunity. That is where the setup effort pays off. Below that, you are complicating your life for nothing.

Verify it works

Never trust a pipeline on its word. Once it is live, compare three volumes for the same day: what sGTM wrote, what the GA4 export received, and what the GA4 interface shows. The control query is simple:

SELECT
  DATE(event_ts) AS day,
  COUNT(*) AS direct_events
FROM `project.dataset.events_direct`
WHERE DATE(event_ts) = CURRENT_DATE() - 1
GROUP BY day;

A gap is normal (ad-blocker immunity and enrichment mean the direct pipeline captures more), but it must be explainable. A gap you cannot account for is a bug in hiding.

Writing directly from sGTM to BigQuery is not the default solution, and that is exactly right. It is the practitioner’s tool for someone who hit a real ceiling or has a real enrichment need. If that is you, you just gained a pipeline with no limit, near real-time, that you own end to end. If it is not, keep the native export and sleep well.