ThinkETL

ThinkETL

2/8/2025, 6:44:34 AM

Do you ever need to pull data into Snowflake from external sources using APIs? 🤔 Snowflake, as a powerful cloud data platform, offers a secure and efficient way to make external API calls directly within its environment. Here’s a quick steps to making external API calls from within Snowflake: ❄️𝐂𝐫𝐞𝐚𝐭𝐞 𝐚 𝐍𝐞𝐭𝐰𝐨𝐫𝐤 𝐑𝐮𝐥𝐞: Define and control the endpoints that can communicate with your environment. 🔗 ❄️𝐂𝐫𝐞𝐚𝐭𝐞 𝐚 𝐒𝐞𝐜𝐫𝐞𝐭: Securely store all your authentication details—usernames, passwords, tokens, etc.—to keep your connections protected. 🔐 ❄️𝐂𝐫𝐞𝐚𝐭𝐞 𝐚𝐧 𝐄𝐱𝐭𝐞𝐫𝐧𝐚𝐥 𝐀𝐜𝐜𝐞𝐬𝐬 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧: Seamlessly combine your network rules and secrets into one powerful configuration. ⚙️ ❄️𝐂𝐫𝐞𝐚𝐭𝐞 𝐚 𝐒𝐧𝐨𝐰𝐟𝐥𝐚𝐤𝐞 𝐒𝐭𝐨𝐫𝐞𝐝 𝐏𝐫𝐨𝐜𝐞𝐝𝐮𝐫𝐞 𝐨𝐫 𝐅𝐮𝐧𝐜𝐭𝐢𝐨𝐧: Leverage the integration and secure credentials to execute API calls directly from your stored procedures or functions. 🚀 Checkout the complete article for complete details: 💬 https://thinketl.com/how-to-make-external-api-calls-from-snowflake/ #ThinkETL #Snowflake #SnowflakeDataCloud #Cloud #API #Python #StoredProcedures

ThinkETL

2/26/2025, 5:56:00 AM

The 𝐂𝐎𝐏𝐘 𝐈𝐍𝐓𝐎 command in Snowflake is a powerful tool for efficiently loading data from staged files into database tables. Key features include: ❄️ 𝑭𝒍𝒆𝒙𝒊𝒃𝒍𝒆 𝑭𝒊𝒍𝒆 𝑯𝒂𝒏𝒅𝒍𝒊𝒏𝒈:  Supports loading from internal and external stages. ❄️ 𝑭𝒊𝒍𝒆 𝑭𝒐𝒓𝒎𝒂𝒕 𝑺𝒖𝒑𝒑𝒐𝒓𝒕:  Works with CSV, JSON, AVRO, ORC, PARQUET, and XML formats. ❄️ 𝑫𝒂𝒕𝒂 𝑺𝒆𝒍𝒆𝒄𝒕𝒊𝒐𝒏 & 𝑻𝒓𝒂𝒏𝒔𝒇𝒐𝒓𝒎𝒂𝒕𝒊𝒐𝒏:  Allows selecting specific columns and applying transformations during load. ❄️ 𝑷𝒂𝒕𝒕𝒆𝒓𝒏 𝑴𝒂𝒕𝒄𝒉𝒊𝒏𝒈 & 𝑭𝒊𝒍𝒆 𝑺𝒆𝒍𝒆𝒄𝒕𝒊𝒐𝒏:  Enables loading specific files using 𝐏𝐀𝐓𝐓𝐄𝐑𝐍 or 𝐅𝐈𝐋𝐄𝐒 options. ❄️ 𝑬𝒓𝒓𝒐𝒓 𝑯𝒂𝒏𝒅𝒍𝒊𝒏𝒈:  Provides multiple strategies like 𝐂𝐎𝐍𝐓𝐈𝐍𝐔𝐄, 𝐒𝐊𝐈𝐏_𝐅𝐈𝐋𝐄, and 𝐀𝐁𝐎𝐑𝐓_𝐒𝐓𝐀𝐓𝐄𝐌𝐄𝐍𝐓. ❄️ 𝑽𝒂𝒍𝒊𝒅𝒂𝒕𝒊𝒐𝒏 𝑴𝒐𝒅𝒆:  Allows checking file integrity before loading using VALIDATION_MODE. ❄️ 𝑴𝒆𝒕𝒂𝒅𝒂𝒕𝒂 𝑴𝒂𝒑𝒑𝒊𝒏𝒈:  Maps metadata columns (e.g., filename, row number) using 𝐈𝐍𝐂𝐋𝐔𝐃𝐄_𝐌𝐄𝐓𝐀𝐃𝐀𝐓𝐀. ❄️ 𝑷𝒆𝒓𝒇𝒐𝒓𝒎𝒂𝒏𝒄𝒆 𝑶𝒑𝒕𝒊𝒎𝒊𝒛𝒂𝒕𝒊𝒐𝒏:  Includes options like 𝐒𝐈𝐙𝐄_𝐋𝐈𝐌𝐈𝐓, 𝐅𝐎𝐑𝐂𝐄, and 𝐌𝐀𝐓𝐂𝐇_𝐁𝐘_𝐂𝐎𝐋𝐔𝐌𝐍_𝐍𝐀𝐌𝐄. ❄️ 𝑭𝒊𝒍𝒆 𝑴𝒂𝒏𝒂𝒈𝒆𝒎𝒆𝒏𝒕:  Supports automatic file removal from stages using 𝐏𝐔𝐑𝐆𝐄. Checkout the complete article for complete details: https://thinketl.com/data-load-using-copy-into-command-in-snowflake #ThinkETL #Snowflake #SnowflakeDataCloud #Cloud #DataLoading