Versions Compared

Old Version 3

changes.mady.by.user Shahul Naveed

Saved on

compared with

New Version 4

changes.mady.by.user Shahul Naveed

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Script Based Install Guide: Azure Synapse for RED 8

This is a guide to installing the WhereScape Enablement Pack for Azure Synapse for WhereScape RED 8.6.1.x 

...

The Enablement Pack Install process is entirely driven by scripts. The below table outlines these scripts, their purpose, and if Run as Administrator is required. 

#

Enablement Pack Setup Scripts

Script Purpose

Run as Admin

Intended Application

1

Setup_Enablement_Pack.ps1

Setup and configure a RED Metadata Repository for Azure Synapse
If RED repository exists then update the repository with 1. Templates 2. Scripts 3. Extended Properties 4. Datatype Mappings 5.UI Configurations

Yes

New and Existing installations

2

install_WslPowershell_Modules.bat

Installs or updates WslPowershell Modules on this machine

Yes

New and Existing installations

3

install_WslPython_Modules.bat

Installs or updates WslPython Modules and required Python libraries on this machine

Yes

New and Existing installations

4

import_powershell_templates.ps1

Imports or updates the Powershell Templates to a RED Metadata Repository. Also includes any Script or Procedure Imports

No*

Existing installations

5

set_default_templates.ps1

Applies the RED Connection defaults in a RED Metadata Repository for Python or Powershell templates

No*

Existing installations


Note

Note that on some systems executing Windows Powershell scripts is disabled by default, see troubleshooting for workarounds

...

Source Enablement Pack Support

Source Pack Name

Supported By Azure Synapse

Supported Features

Amazon S3

Yes

Install Powershell package:
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3bdadc06-57df-4f02-8b8a-85ba03d06a64"><ac:plain-text-body><![CDATA[* [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
]]></ac:plain-text-body></ac:structured-macro>

  • Install-Module -Name AWSPowerShell.NetCore
  • Install-Module -Name AWS.Tools.Installer

Azure Data Lake Storage Gen2

Yes

Load file from Azure Data Lake Storage Gen2 bucket to Azure Synapse
Install Python package:

  •  pip install azure-storage-file-datalake

    Required Properties:
  • Azure Data Lake Storage Gen2 Account Name
  • Azure Data Lake Storage Gen2 Access Key
  • Azure Data Lake Storage Gen2 SAS Token
  • Azure Data Lake Storage Gen2 Container Name
  • Azure Data Lake Storage Gen2 Directory Name (Created in Container)

Google Drive

No

None

Google Cloud

Yes

Please refer the official link for steps to download gsutil: {+}https://cloud.google.com/storage/docs/gsutil_install+Image Modified

Windows Parser

1.CSV
2.Excel
3.JSON
4.XML
5.AVRO
6.ORC
7.PARQUET

Refer to Windows Parser Guide.
Load Template ,Source Properties will have option to select parser type to load the files.
Image Modified


Note

File Types: ORC
For copying ORC data to Synapse, the complex data types are not currently supported (STRUCT, MAP, LIST, UNION), this is a current limitation in Azure Synapse. 

...