Questions tagged [google-bigquery]
Google BigQuery is a Google Cloud Platform product providing serverless queries of petabyte-scale data sets using SQL. BigQuery provides multiple read-write pipelines, and enables data analytics that transform how businesses analyze data.
google-bigquery
26,202
questions
1
vote
0
answers
30
views
For a BQ SQL table, calculate the number of times the value in a given column switches in a given time period, partitioned by an ID value
Say I have a BQ table called employee_entries that is created and populated by the following SQL statements:
CREATE OR REPLACE TABLE `my-project.my_dataset.employee_entries` (
employee_id STRING,
...
-2
votes
0
answers
18
views
Can a subquery be used without an alias in bigquery
In bigquery I ran the following query and it did not throw any error:
SELECT *
From (
SELECT SUBSTR(FileRecordId,1,3) x
From adept-box-428408-s6.tboy.Food_events
)
This query worked fine without ...
- 109
0
votes
0
answers
9
views
Can this be turned into an alter table function to remove the pulled duplicates?
SELECT ride_id,
rideable_type,
started_at,
ended_at,
start_station_name,
start_station_id,
end_station_name,
start_lat,
start_lng,
end_lat,
end_lng,
member_casual,
FROM (
SELECT ride_id ,
...
1
vote
2
answers
30
views
Filtering for unique pairs in a SQL join- simplify a working query
I have data for event registrations where users have an account on the website (user_id), submit a form for registration (nametag), and pay via stripe (order_id).
Each registration must have a unique ...
- 992
0
votes
0
answers
9
views
Using the Confluent BigQuery sink Kafka connector, why are my messages/records not being written to new BQ tables, but existing tables are?
I am running Kafka Connect locally using Docker, but also in AWS MSK. Both are producing the same results.
My connector configuration is as follows:
{
"name": "connector-name",
&...
- 440
1
vote
1
answer
10
views
Creating a table in BigQuery with mixed data types
I have a table in BigQuery I'm trying to create in BigQuery that looks as such:
state
action_date
Alabama
None
Alaska
10/21/24
Delaware
10/28/24
District of Columbia
10/25/24
Florida
10/26/24
...
- 656
0
votes
0
answers
30
views
Error Loading ORC Files into BigQuery: "Invalid timestamp value" after 17th July 2024
I've been successfully fetching ORC files from an Azure Blob Storage to a GCS bucket using the Storage Transfer Service. This process handles around 100 files. After loading the files into GCS, I use ...
- 37
0
votes
0
answers
12
views
bigquery data transfer service is not syncing with Teradata extraction in Python
The need is to extract the data from Teradata to bigquery using bigquery data transfer service BQ DTS in my python code.
what I faced the problem here is I am unable to pass teradata connection ...
- 1
0
votes
1
answer
30
views
BigQuery Error about SELECT list expression references column date which is neither grouped nor aggregated at []
Recently I have been learning SQL and when I tried the following code I got an error, I would like to know what's wrong with the code
SELECT
CAST(date AS date) AS date_only,
CAST(purchase_price AS ...
-1
votes
1
answer
19
views
Extract part of text from a variable declared Bigquery regexp_extract
I have a declared variable in bigquery with words that I want to identify in a table
DECLARE words_to_find DEFAULT r'(CASA|PERRO|CORAZON|GALLETA)';
WITH tabla AS
( select 1 as row_id, 'QUIERO IR ...
- 1
0
votes
0
answers
11
views
How to handle new fields in a STRUCT when dynamically adding using union on all tables in BigQuery
Process
In BigQuery, Google Analytics data is linked to a BigQuery project. Each site corresponds with a dataset, and each day an events_* table is added to that dataset. My query dynamically iterates ...
- 423
0
votes
1
answer
37
views
How to reduce decimal places to just 2?
I have this query in BigQuery
SELECT Id,
AVG(VeryActiveMinutes) AS avg_very_active,
SUM(VeryActiveMinutes) AS total_very_active,
SUM(SedentaryMinutes) AS total_sedentary
FROM `...
0
votes
0
answers
40
views
DETERMINISTIC_DECRYPT_STRING failed: Keyset deserialization failed: Error reading keyset data: Could not parse the input stream as a Keyset-proto
I have a data encryption key encrypted (DEK) that was used to encrypt some values using the algorithm AES256 and the encrypted values are in a BigQuery table.
I am using the KMS provided by Google to ...
- 59
0
votes
1
answer
12
views
Creating bigquery tables with expiration option leads to unexpected expiration time
I'm creating a table in GBQ with an expiration timestamp as following:
CREATE OR REPLACE TABLE
`myproject.mydataset.testtable`
OPTIONS (expiration_timestamp=TIMESTAMP(DATE_ADD(CURRENT_DATE(), ...
- 1,646
1
vote
0
answers
56
views
BigQuery Data transfer crashes with error INVALID_ARGUMENT: Creating table with tags is not supported
I have crashlytics being synchronized in BigQuery and I have a Data transfer job configured to copy its data from US to EU region. It was working great for years and now I got an error, since 1 week.
...
- 73