Transform data with data manipulation language (DML)

The BigQuery data manipulation language (DML) enables you to update, insert, and delete data from your BigQuery tables.

You can execute DML statements just as you would a SELECT statement, with the following conditions:

For a list of BigQuery DML statements and examples of how to use them, see Data manipulation language statements in GoogleSQL.

Limitations

Concurrent jobs

BigQuery manages the concurrency of DML statements that add, modify, or delete rows in a table.

INSERT DML concurrency

During any 24 hour period, the first 1500 INSERT statements run immediately after they are submitted. After this limit is reached, the concurrency of INSERT statements that write to a table is limited to 10. Additional INSERT statements are added to a PENDING queue. Up to 100 INSERT statements can be queued against a table at any given time. When an INSERT statement completes, the next INSERT statement is removed from the queue and run.

UPDATE, DELETE, MERGE DML concurrency

The UPDATE, DELETE, and MERGE DML statements are called mutating DML statements. If you submit one or more mutating DML statements on a table while other mutating DML jobs on it are still running (or pending), BigQuery runs up to 2 of them concurrently, after which up to 20 are queued as PENDING. When a previously running job finishes, the next pending job is dequeued and run. Queued mutating DML statements share a per-table queue with maximum length 20. Additional statements past the maximum queue length for each table fail with the error message: Resources exceeded during query execution: Too many DML statements outstanding against table PROJECT_ID:TABLE, limit is 20.

Interactive priority DML jobs that are queued for more than 6 hours fail with the following error message:

DML statement has been queued for too long

DML statement conflicts

Mutating DML statements that run concurrently on a table cause DML statement conflicts when the statements try to mutate the same partition. The statements succeed as long as they don't modify the same partition. BigQuery tries to rerun failed statements up to three times.

Pricing

For information on DML pricing, see Data Manipulation Language pricing on the Pricing page.

Best practices

For best performance, Google recommends the following patterns:

For best practices to optimize query performance, see Introduction to optimizing query performance.

What's next