Hi alll, i am working on

tap-shopify

and my target is

target-redshift

, i have a query related to column size of redshift. i just want to manipulate or change the columns size of my table created in redshift, by default redhshift column size is 1000, but it’s nit efficient for every columns i have my columns size which i want to set.. is there any way to do this?

I haven’t found/tried looking for ways to set column sizes automatically in redshift via the target, but I have had some success letting the target create them initially based on some sample data and then manually adjusting them after the fact. I don’t know that that’s the best option, but it’s at least an option

is there any example?

When we looked into redshift, you can change it (I forget how) but you can't make these bigger than 65535. We were discussing alternatives like not putting free-form text in the warehouse but I think we're just going to use Snowflake instead

Yep, that’s right - the only way I’ve been able to adjust individual columns is via direct

ALTER

commands in redshift itself. As long as the column names don’t change, the target doesn’t seem to care

try the pipelinewise redshift target, I think it does a better job with tigher column lengths from memory

I have found that the pipelinewise redshift target does in fact deal with column lengths way better than target-redshift. However, an issue with the pipelinewise variant is it handles denesting columns poorly.

I’ve had a check and it seems that pipelinewise redshift still jsut sets the default to 10000