Questions tagged [partitioning]
Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.
partitioning
3,207
questions
0
votes
1
answer
25
views
How to partition data in a warp based on a predicate so all keep items are consecutive
I have a warp full of data, some of which I want to keep and some I want to discard.
I want to store the keep items in contiguous memory.
For example, say I only want to keep prime numbers
input ...
- 75.5k
-2
votes
0
answers
17
views
Linux fedora option won't appear on grub boot menu after new fedora (additional) instalation [closed]
Recently I had my fedora (39) running ok, every time I booted the pc grub boot menu jumped in asking which distro I wanted to run. Everything ok.
Then I created a new disk partition and installed a ...
- 25
-4
votes
0
answers
54
views
how to write this row mapper in better way [closed]
'm working on a Spring Batch application and have implemented a custom Partitioner to divide processing across multiple threads. I'm having trouble writing a JUnit 4 test for my partition method. I've ...
1
vote
0
answers
32
views
How to partition data in Spark when reading data from a MySQL table with string type primary key?
I'm reading data from a MySQL table in Spark. The table structure is like this:
CREATE TABLE my_table (
id varchar(64),
content varchar(64),
primary key id(id)
)
My code in Spark is like ...
- 832
-2
votes
2
answers
42
views
How to find percentage on in SQL?
I have data like this:
I want to find the percentage as a third column for each row, No- 549/(549+342)
Yes - 342/(549+342)
I have tried select Survived, cnt, round(100*(cnt/sum(cnt)), 2) as prcntage ...
- 1,842
0
votes
1
answer
67
views
Is changing date partitionning granularity a breaking change?
In Bigquery, suppose I create a table and partition it by a date column "mydate" with a "DAY" granularity.
Using DBT, this can be done using :
partition_by = {
"...
- 11
0
votes
0
answers
35
views
Unable to sync non-partitioned Hudi table with BigQuery
I'm trying to to write my structured streaming data to Apache Hudi in a non-partitioned table and then sync it with BigQuery. But even though it is a new table and I've set no partitioning ...
0
votes
1
answer
52
views
Create Partitions in External Tables in Azure Synapse SQL Database (Serverless)
I want to create an external table with partition columns based on year, month and day. I want to be able to query through this data in an optimized way. The queries could involve using a range of ...
- 1
1
vote
0
answers
41
views
issue in PostgrelSQL 15.3 with partition pruning
I have two tables partitioned by range on column day,(this is an oversimplification for a more complex design), tables appear similar but day field refers to completly different concept than day in ...
1
vote
0
answers
29
views
BIG Query recursive CTE or other possible solution
I have a table with 10 million unique Product_ID. This Unique Ids are being tested by the hr and the metric has a [good : -1, neutral : 0, bad : 1], I call this column hr_status. I want to develop a ...
0
votes
1
answer
65
views
Google Pubsub Cloud Storage subscription to combine messages into same avro file
I have a Google Pubsub Topic with no schema enforcement (nor I would like to have a schema enforcement)
I have a Google Pubsub Cloud Storage Subscription setup to:
flush messages to GCS (Google Cloud ...
- 4,791
0
votes
0
answers
23
views
Can I use hash partitioning on a non-primary key column in a PostgreSQL table?
I am trying to replace an existing table with a partitioned version of that table. I want the schema of the new partitioned table to exactly match the existing non-partitioned table so that I can ...
-1
votes
1
answer
104
views
std::partition_copy: what happens when the d_first_true output range overlaps with the input range? [duplicate]
for example:
int original_range[] = {1, 2, 3, 4, 5, 6, 7};
int copy_here[7];
std::partition_copy(std::begin(original_range), std::end(original_range),
std::begin(original_range), ...
- 139
2
votes
2
answers
96
views
Optimize query on partitioned table without partitioning key in the WHERE clause
We are trying to optimize a query to a partitioned table, the query looks something like this:
SELECT col1, col2
FROM partitioned_table
WHERE profile_id = '00000000-0000-0000-0000-000000000000'
AND ...
- 23
0
votes
0
answers
35
views
Cold data migration with sliding window stored procedure for partitioned tables in SQL Server
I am trying to write a sliding window stored procedure in SQL Server in order to migrate cold data to a cold storage. To do this, I planned on creating an "Archiving" table, that will store ...
