End of Line Matters for Shell Script in CICD Pipeline

End of Line Matters for Shell Script in CICD Pipeline

You may get an error or failed pipeline if the end of line of a shell script file is CRLF (\r\n) not LF (\n) 

LF

CRLF

In VS Code, you may convert the End of Line from CRLF to LF. Or you may change the default setting

SnowPro Core Certification: Snowflake Data Cloud Features & Architecture （1/8）

SnowPro Core Certification: Snowflake Data Cloud Features & Architecture （1/8）

Snowflake Architecture

#cloud services layer

The cloud services layer is a collection of services that coordinate activities across Snowflake. These services tie together all of the different components of Snowflake in order to process user requests, from login to query dispatch. The cloud services layer also runs on compute instances provisioned by Snowflake from the cloud provider.

The cloud services layer is responsible for query planning and optimization.

Virtual warehouses perform the query processing, but once a result set has been created, it is stored in the query result cache, which is part of the cloud services layer.

Snowflake's cloud services layer is its brain and is a reliable, always-on service. Snowflake accounts are only accessible via cloud services. All requests to Snowflake, whether via the Snowflake web UI or SnowSQL, travel through this layer.

Services managed in this layer include:

• Authentication

• Infrastructure management

• Metadata management

• Query parsing and optimization

• Access control

https://docs.snowflake.com/en/user-guide/intro-key-concepts#cloud-services

#Query Processing

The query processing layer is the compute layer through which queries and data processing jobs are executed on the stored data. The compute layer can have multiple clusters for a given Snowflake instance simultaneously. The compute engines in Snowflake are known as virtual warehouses.

Virtual warehouses perform the query processing, but once a result set has been created, it is stored in the query result cache, which is part of the cloud services layer.  

Query execution is performed in the processing layer. Snowflake processes queries using “virtual warehouses”. Each virtual warehouse is an MPP compute cluster composed of multiple compute nodes allocated by Snowflake from a cloud provider.

Each virtual warehouse is an independent compute cluster that does not share compute resources with other virtual warehouses. As a result, each virtual warehouse has no impact on the performance of other virtual warehouses.

#Database Storage

When data is loaded into Snowflake, Snowflake reorganizes that data into its internal optimized, compressed, columnar format. Snowflake stores this optimized data in cloud storage.

Snowflake's shared storage layer resides on low-cost object cloud storage. Snowflake currently supports AWS S3 storage, Azure Blob Storage, and Google Cloud Storage for data storage.

Snowflake manages all aspects of how this data is stored — the organization, file size, structure, compression, metadata, statistics, and other aspects of data storage are handled by Snowflake. The data objects stored by Snowflake are not directly visible nor accessible by customers; they are only accessible through SQL query operations run using Snowflake.

Snowflake Data Cloud Features

The benefits of The Data Cloud are Access, Governance, and Action (AGA).

• Access means that organizations can easily discover data and share it internally or with third parties without regard to geographical location.

• Governance is about setting policies and rules and protecting the data in a way that can unlock new value and collaboration while maintaining the highest levels of security and compliance.

• Action means you can empower every part of your business with data to build better products, make faster decisions, create new revenue streams, and realize the value of your greatest untapped asset, your data.

SnowPro Core Certification: 开篇 （0/8）

SnowPro Core Certification: 开篇 （0/8）

SnowPro Core Certification

https://learn.snowflake.com/en/certifications/snowpro-core/

What It Is

The SnowPro® Core Certification is designed for individuals who would like to demonstrate their knowledge of the Snowflake Data Cloud.

The candidate has a thorough knowledge of:

•Data Loading and Transformation in Snowflake.
•Virtual Warehouse Performance and Concurrency
•DDL and DML Queries
•Using Semi-Structured and Unstructured Data
•Cloning and Time Travel
•Data Sharing
•Snowflake Account Structure and Management

Exam Format

Total Number of Questions: 100

Question Types: Multiple Select, Multiple Choice

Time Limit: 115 minutes

Languages: English & Japanese

Registration fee: $175 USD (不便宜）

Passing Score: 750 + Scaled Scoring from 0 - 1000

Unscored Content: Exams may include unscored items to gather statistical information for future use. These items are not identified on the form and do not impact your score, and additional time is factored into account for this content.

Domain

This exam guide includes test domains, weightings, and objectives. It is not a comprehensive listing of all the content that will be presented in this examination. The table below lists the main content domains and their weightings. 

Domain                                                                         Domain Weightings on Exams

1.0 Snowflake Data Cloud Features & Architecture 25%

2.0 Account Access and Security                                 20%

3.0 Performance Concepts                                         15%

4.0 Data Loading and Unloading                                 10%

5.0 Data Transformations                                         20%

6.0 Data Protection and Data Sharing                         10%

Note:

1. 据snowflake官网介绍snowflake文档100%覆盖考题（根据我的考试经验不是这样的，有些题需要深入思考。）

2. 如果有充足的时间和良好的记忆力，通读文档通过考试应该没有问题。

3. 如果没有太多时间和良好的记忆力，那就期待我划重点吧！我在后续文章中会按照领域进行总结、举例和探讨。