Python

Last updated Feb 28, 2024

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically-typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and Functional Programming.

Python is the de facto standard for Data Engineering next to SQL.

I’d learn how to write Functional Data Engineering code within python. My favorite is dagster, which forces you to write functional blocks of code with superior features. Python and Functional Programming is also an excellent read in that direction.

Python is one of the most accessible programming l code within Python. My favorite is Dagster, which forces you to write functional blocks of code with superior features—coming from a more SQL, T-SQL, and PL-SQL background. As a data engineer, I’d say you’d not expect to write perfect code; it’s better to know the Big-O annotation to avoid long-running data pipelines, even if your code doesn’t look the prettiest. Static types such as mypy might be another good one to know, as it will detect errors pre-runtime, which is the biggest problem of Python.

Breaking Packages in Python | Dagster Blog

References: The growth of major programming languages