airbyte 0.7.0-beta2, Mar 7 25

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

airbyte.DestinationS3Glue

Explore with Pulumi AI

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

Example Usage

Coming soon!

Coming soon!

Coming soon!

Coming soon!

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.airbyte.DestinationS3Glue;
import com.pulumi.airbyte.DestinationS3GlueArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs;
import com.pulumi.airbyte.inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var myDestinationS3glue = new DestinationS3Glue("myDestinationS3glue", DestinationS3GlueArgs.builder()
            .configuration(DestinationS3GlueConfigurationArgs.builder()
                .access_key_id("A012345678910EXAMPLE")
                .file_name_pattern("{date}")
                .format(DestinationS3GlueConfigurationFormatArgs.builder()
                    .jsonLinesNewlineDelimitedJson(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs.builder()
                        .compression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs.builder()
                            .gzip(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs.builder()
                                .compressionType("GZIP")
                                .build())
                            .noCompression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs.builder()
                                .compressionType("No Compression")
                                .build())
                            .build())
                        .flattening("No flattening")
                        .formatType("JSONL")
                        .build())
                    .build())
                .glue_database("airbyte_database")
                .glue_serialization_library("org.openx.data.jsonserde.JsonSerDe")
                .s3_bucket_name("airbyte_sync")
                .s3_bucket_path("data_sync/test")
                .s3_bucket_region("us-gov-west-1")
                .s3_endpoint("http://localhost:9000")
                .s3_path_format(String.format("%s/%s/%s_%s_%s_%s_", NAMESPACE,STREAM_NAME,YEAR,MONTH,DAY,EPOCH))
                .secret_access_key("a012345678910ABCDEFGH/AbCdEfGhEXAMPLEKEY")
                .build())
            .definitionId("7a9c505b-1990-4b38-b243-1c254ff78c36")
            .workspaceId("e7df4138-2176-42e6-aa11-ed72fe60b365")
            .build());

    }
}

resources:
  myDestinationS3glue:
    type: airbyte:DestinationS3Glue
    properties:
      configuration:
        access_key_id: A012345678910EXAMPLE
        file_name_pattern: '{date}'
        format:
          jsonLinesNewlineDelimitedJson:
            compression:
              gzip:
                compressionType: GZIP
              noCompression:
                compressionType: No Compression
            flattening: No flattening
            formatType: JSONL
        glue_database: airbyte_database
        glue_serialization_library: org.openx.data.jsonserde.JsonSerDe
        s3_bucket_name: airbyte_sync
        s3_bucket_path: data_sync/test
        s3_bucket_region: us-gov-west-1
        s3_endpoint: http://localhost:9000
        s3_path_format: ${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_
        secret_access_key: a012345678910ABCDEFGH/AbCdEfGhEXAMPLEKEY
      definitionId: 7a9c505b-1990-4b38-b243-1c254ff78c36
      workspaceId: e7df4138-2176-42e6-aa11-ed72fe60b365

Create DestinationS3Glue Resource

Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

Constructor syntax

new DestinationS3Glue(name: string, args: DestinationS3GlueArgs, opts?: CustomResourceOptions);

@overload
def DestinationS3Glue(resource_name: str,
                      args: DestinationS3GlueArgs,
                      opts: Optional[ResourceOptions] = None)

@overload
def DestinationS3Glue(resource_name: str,
                      opts: Optional[ResourceOptions] = None,
                      configuration: Optional[DestinationS3GlueConfigurationArgs] = None,
                      workspace_id: Optional[str] = None,
                      definition_id: Optional[str] = None,
                      name: Optional[str] = None)

func NewDestinationS3Glue(ctx *Context, name string, args DestinationS3GlueArgs, opts ...ResourceOption) (*DestinationS3Glue, error)

public DestinationS3Glue(string name, DestinationS3GlueArgs args, CustomResourceOptions? opts = null)

public DestinationS3Glue(String name, DestinationS3GlueArgs args)
public DestinationS3Glue(String name, DestinationS3GlueArgs args, CustomResourceOptions options)

type: airbyte:DestinationS3Glue
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

Parameters

name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

resource_name str: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts ResourceOptions: Bag of options to control resource's behavior.

ctx Context: Context object for the current deployment.
name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts ResourceOption: Bag of options to control resource's behavior.

name string: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
opts CustomResourceOptions: Bag of options to control resource's behavior.

name String: The unique name of the resource.
args DestinationS3GlueArgs: The arguments to resource properties.
options CustomResourceOptions: Bag of options to control resource's behavior.

Constructor example

The following reference example uses placeholder values for all input properties.

var destinationS3GlueResource = new Airbyte.DestinationS3Glue("destinationS3GlueResource", new()
{
    Configuration = new Airbyte.Inputs.DestinationS3GlueConfigurationArgs
    {
        Format = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatArgs
        {
            JsonLinesNewlineDelimitedJson = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs
            {
                Compression = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs
                {
                    Gzip = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs
                    {
                        CompressionType = "string",
                    },
                    NoCompression = new Airbyte.Inputs.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs
                    {
                        CompressionType = "string",
                    },
                },
                Flattening = "string",
                FormatType = "string",
            },
        },
        GlueDatabase = "string",
        S3BucketName = "string",
        S3BucketPath = "string",
        AccessKeyId = "string",
        FileNamePattern = "string",
        GlueSerializationLibrary = "string",
        S3BucketRegion = "string",
        S3Endpoint = "string",
        S3PathFormat = "string",
        SecretAccessKey = "string",
    },
    WorkspaceId = "string",
    DefinitionId = "string",
    Name = "string",
});

example, err := airbyte.NewDestinationS3Glue(ctx, "destinationS3GlueResource", &airbyte.DestinationS3GlueArgs{
Configuration: &.DestinationS3GlueConfigurationArgs{
Format: &.DestinationS3GlueConfigurationFormatArgs{
JsonLinesNewlineDelimitedJson: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs{
Compression: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs{
Gzip: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs{
CompressionType: pulumi.String("string"),
},
NoCompression: &.DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs{
CompressionType: pulumi.String("string"),
},
},
Flattening: pulumi.String("string"),
FormatType: pulumi.String("string"),
},
},
GlueDatabase: pulumi.String("string"),
S3BucketName: pulumi.String("string"),
S3BucketPath: pulumi.String("string"),
AccessKeyId: pulumi.String("string"),
FileNamePattern: pulumi.String("string"),
GlueSerializationLibrary: pulumi.String("string"),
S3BucketRegion: pulumi.String("string"),
S3Endpoint: pulumi.String("string"),
S3PathFormat: pulumi.String("string"),
SecretAccessKey: pulumi.String("string"),
},
WorkspaceId: pulumi.String("string"),
DefinitionId: pulumi.String("string"),
Name: pulumi.String("string"),
})

var destinationS3GlueResource = new DestinationS3Glue("destinationS3GlueResource", DestinationS3GlueArgs.builder()
    .configuration(DestinationS3GlueConfigurationArgs.builder()
        .format(DestinationS3GlueConfigurationFormatArgs.builder()
            .jsonLinesNewlineDelimitedJson(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs.builder()
                .compression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs.builder()
                    .gzip(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs.builder()
                        .compressionType("string")
                        .build())
                    .noCompression(DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs.builder()
                        .compressionType("string")
                        .build())
                    .build())
                .flattening("string")
                .formatType("string")
                .build())
            .build())
        .glueDatabase("string")
        .s3BucketName("string")
        .s3BucketPath("string")
        .accessKeyId("string")
        .fileNamePattern("string")
        .glueSerializationLibrary("string")
        .s3BucketRegion("string")
        .s3Endpoint("string")
        .s3PathFormat("string")
        .secretAccessKey("string")
        .build())
    .workspaceId("string")
    .definitionId("string")
    .name("string")
    .build());

destination_s3_glue_resource = airbyte.DestinationS3Glue("destinationS3GlueResource",
    configuration={
        "format": {
            "json_lines_newline_delimited_json": {
                "compression": {
                    "gzip": {
                        "compression_type": "string",
                    },
                    "no_compression": {
                        "compression_type": "string",
                    },
                },
                "flattening": "string",
                "format_type": "string",
            },
        },
        "glue_database": "string",
        "s3_bucket_name": "string",
        "s3_bucket_path": "string",
        "access_key_id": "string",
        "file_name_pattern": "string",
        "glue_serialization_library": "string",
        "s3_bucket_region": "string",
        "s3_endpoint": "string",
        "s3_path_format": "string",
        "secret_access_key": "string",
    },
    workspace_id="string",
    definition_id="string",
    name="string")

const destinationS3GlueResource = new airbyte.DestinationS3Glue("destinationS3GlueResource", {
    configuration: {
        format: {
            jsonLinesNewlineDelimitedJson: {
                compression: {
                    gzip: {
                        compressionType: "string",
                    },
                    noCompression: {
                        compressionType: "string",
                    },
                },
                flattening: "string",
                formatType: "string",
            },
        },
        glueDatabase: "string",
        s3BucketName: "string",
        s3BucketPath: "string",
        accessKeyId: "string",
        fileNamePattern: "string",
        glueSerializationLibrary: "string",
        s3BucketRegion: "string",
        s3Endpoint: "string",
        s3PathFormat: "string",
        secretAccessKey: "string",
    },
    workspaceId: "string",
    definitionId: "string",
    name: "string",
});

type: airbyte:DestinationS3Glue
properties:
    configuration:
        accessKeyId: string
        fileNamePattern: string
        format:
            jsonLinesNewlineDelimitedJson:
                compression:
                    gzip:
                        compressionType: string
                    noCompression:
                        compressionType: string
                flattening: string
                formatType: string
        glueDatabase: string
        glueSerializationLibrary: string
        s3BucketName: string
        s3BucketPath: string
        s3BucketRegion: string
        s3Endpoint: string
        s3PathFormat: string
        secretAccessKey: string
    definitionId: string
    name: string
    workspaceId: string

DestinationS3Glue Resource Properties

To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

Inputs

In Python, inputs that are objects can be passed either as argument classes or as dictionary literals.

The DestinationS3Glue resource accepts the following input properties:

Configuration DestinationS3GlueConfiguration
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

Configuration DestinationS3GlueConfigurationArgs
WorkspaceId string
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
Name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfiguration
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfiguration
workspaceId string
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name string: Name of the destination e.g. dev-mysql-instance.

configuration DestinationS3GlueConfigurationArgs
workspace_id str
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name str: Name of the destination e.g. dev-mysql-instance.

configuration Property Map
workspaceId String
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
name String: Name of the destination e.g. dev-mysql-instance.

Outputs

All input properties are implicitly available as output properties. Additionally, the DestinationS3Glue resource produces the following output properties:

CreatedAt double
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.

CreatedAt float64
DestinationId string
DestinationType string
Id string: The provider-assigned unique ID for this managed resource.

createdAt Double
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.

createdAt number
destinationId string
destinationType string
id string: The provider-assigned unique ID for this managed resource.

created_at float
destination_id str
destination_type str
id str: The provider-assigned unique ID for this managed resource.

createdAt Number
destinationId String
destinationType String
id String: The provider-assigned unique ID for this managed resource.

Look up Existing DestinationS3Glue Resource

Get an existing DestinationS3Glue resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

public static get(name: string, id: Input<ID>, state?: DestinationS3GlueState, opts?: CustomResourceOptions): DestinationS3Glue

@staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        configuration: Optional[DestinationS3GlueConfigurationArgs] = None,
        created_at: Optional[float] = None,
        definition_id: Optional[str] = None,
        destination_id: Optional[str] = None,
        destination_type: Optional[str] = None,
        name: Optional[str] = None,
        workspace_id: Optional[str] = None) -> DestinationS3Glue

func GetDestinationS3Glue(ctx *Context, name string, id IDInput, state *DestinationS3GlueState, opts ...ResourceOption) (*DestinationS3Glue, error)

public static DestinationS3Glue Get(string name, Input<string> id, DestinationS3GlueState? state, CustomResourceOptions? opts = null)

public static DestinationS3Glue get(String name, Output<String> id, DestinationS3GlueState state, CustomResourceOptions options)

resources:  _:    type: airbyte:DestinationS3Glue    get:      id: ${id}

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

resource_name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

name: The unique name of the resulting resource.
id: The unique provider ID of the resource to lookup.
state: Any extra arguments used during the lookup.
opts: A bag of options that control this resource's behavior.

The following state arguments are supported:

Configuration DestinationS3GlueConfiguration
CreatedAt double
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
WorkspaceId string

Configuration DestinationS3GlueConfigurationArgs
CreatedAt float64
DefinitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
DestinationId string
DestinationType string
Name string: Name of the destination e.g. dev-mysql-instance.
WorkspaceId string

configuration DestinationS3GlueConfiguration
createdAt Double
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
workspaceId String

configuration DestinationS3GlueConfiguration
createdAt number
definitionId string: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId string
destinationType string
name string: Name of the destination e.g. dev-mysql-instance.
workspaceId string

configuration DestinationS3GlueConfigurationArgs
created_at float
definition_id str: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destination_id str
destination_type str
name str: Name of the destination e.g. dev-mysql-instance.
workspace_id str

configuration Property Map
createdAt Number
definitionId String: The UUID of the connector definition. One of configuration.destinationType or definitionId must be provided. Requires replacement if changed.
destinationId String
destinationType String
name String: Name of the destination e.g. dev-mysql-instance.
workspaceId String

Supporting Types

DestinationS3GlueConfiguration, DestinationS3GlueConfigurationArgs

Format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
GlueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
S3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
S3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
AccessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
FileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
GlueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
S3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
S3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
S3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
SecretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

Format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
GlueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
S3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
S3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
AccessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
FileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
GlueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
S3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
S3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
S3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
SecretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glueDatabase String: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName String: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath String: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId String: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern String: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary String: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion String: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint String: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat String: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey String: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glueDatabase string: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName string: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath string: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId string: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern string: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary string: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion string: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint string: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat string: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey string: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format DestinationS3GlueConfigurationFormat: Format of the data output. See \n\nhere\n\n for more details
glue_database str: Name of the glue database for creating the tables, leave blank if no integration
s3_bucket_name str: The name of the S3 bucket. Read more \n\nhere\n\n.
s3_bucket_path str: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
access_key_id str: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
file_name_pattern str: The pattern allows you to set the file-name format for the S3 staging file(s)
glue_serialization_library str: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3_bucket_region str: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3_endpoint str: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3_path_format str: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secret_access_key str: The corresponding secret to the access key ID. Read more \n\nhere\n\n

format Property Map: Format of the data output. See \n\nhere\n\n for more details
glueDatabase String: Name of the glue database for creating the tables, leave blank if no integration
s3BucketName String: The name of the S3 bucket. Read more \n\nhere\n\n.
s3BucketPath String: Directory under the S3 bucket where data will be written. Read more \n\nhere\n\n
accessKeyId String: The access key ID to access the S3 bucket. Airbyte requires Read and Write permissions to the given bucket. Read more \n\nhere\n\n.
fileNamePattern String: The pattern allows you to set the file-name format for the S3 staging file(s)
glueSerializationLibrary String: The library that your query engine will use for reading and writing data in your lake. Default: "org.openx.data.jsonserde.JsonSerDe"; must be one of ["org.openx.data.jsonserde.JsonSerDe", "org.apache.hive.hcatalog.data.JsonSerDe"]
s3BucketRegion String: The region of the S3 bucket. See \n\nhere\n\n for all region codes. Default: ""; must be one of ["", "af-south-1", "ap-east-1", "ap-northeast-1", "ap-northeast-2", "ap-northeast-3", "ap-south-1", "ap-south-2", "ap-southeast-1", "ap-southeast-2", "ap-southeast-3", "ap-southeast-4", "ca-central-1", "ca-west-1", "cn-north-1", "cn-northwest-1", "eu-central-1", "eu-central-2", "eu-north-1", "eu-south-1", "eu-south-2", "eu-west-1", "eu-west-2", "eu-west-3", "il-central-1", "me-central-1", "me-south-1", "sa-east-1", "us-east-1", "us-east-2", "us-gov-east-1", "us-gov-west-1", "us-west-1", "us-west-2"]
s3Endpoint String: Your S3 endpoint url. Read more \n\nhere\n\n. Default: ""
s3PathFormat String: Format string on how data will be organized inside the S3 bucket directory. Read more \n\nhere\n\n
secretAccessKey String: The corresponding secret to the access key ID. Read more \n\nhere\n\n

DestinationS3GlueConfigurationFormat, DestinationS3GlueConfigurationFormatArgs

JsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

JsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

json_lines_newline_delimited_json DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson

jsonLinesNewlineDelimitedJson Property Map

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs

Compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
Flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
FormatType string: Default: "JSONL"; must be "JSONL"

Compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
Flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
FormatType string: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening String: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType String: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening string: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType string: Default: "JSONL"; must be "JSONL"

compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening str: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
format_type str: Default: "JSONL"; must be "JSONL"

compression Property Map: Whether the output files should be compressed. If compression is selected, the output filename will have an extra extension (GZIP: ".jsonl.gz").
flattening String: Whether the input json data should be normalized (flattened) in the output JSON Lines. Please refer to docs for details. Default: "Root level flattening"; must be one of ["No flattening", "Root level flattening"]
formatType String: Default: "JSONL"; must be "JSONL"

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs

Gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
NoCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

Gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
NoCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
noCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
noCompression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip
no_compression DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression

gzip Property Map
noCompression Property Map

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs

CompressionType string: Default: "GZIP"; must be "GZIP"

CompressionType string: Default: "GZIP"; must be "GZIP"

compressionType String: Default: "GZIP"; must be "GZIP"

compressionType string: Default: "GZIP"; must be "GZIP"

compression_type str: Default: "GZIP"; must be "GZIP"

compressionType String: Default: "GZIP"; must be "GZIP"

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs

CompressionType string: Default: "No Compression"; must be "No Compression"

CompressionType string: Default: "No Compression"; must be "No Compression"

compressionType String: Default: "No Compression"; must be "No Compression"

compressionType string: Default: "No Compression"; must be "No Compression"

compression_type str: Default: "No Compression"; must be "No Compression"

compressionType String: Default: "No Compression"; must be "No Compression"

Import

$ pulumi import airbyte:index/destinationS3Glue:DestinationS3Glue my_airbyte_destination_s3_glue ""

To learn more about importing existing cloud resources, see Importing resources.

Package Details

Repository: airbyte airbytehq/terraform-provider-airbyte
License
Notes: This Pulumi package is based on the airbyte Terraform Provider.

airbyte 0.7.0-beta2 published on Friday, Mar 7, 2025 by airbytehq

airbytehq/terraform-provider-airbyte

airbyte.DestinationS3Glue

On this page

On this page

Example Usage

Create DestinationS3Glue Resource

Constructor syntax

Parameters

Constructor example

DestinationS3Glue Resource Properties

Inputs

Outputs

Look up Existing DestinationS3Glue Resource

Supporting Types

DestinationS3GlueConfiguration, DestinationS3GlueConfigurationArgs

DestinationS3GlueConfigurationFormat, DestinationS3GlueConfigurationFormatArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJson, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzip, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionGzipArgs

DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompression, DestinationS3GlueConfigurationFormatJsonLinesNewlineDelimitedJsonCompressionNoCompressionArgs

Import

Package Details

On this page

On this page