In PySpark, when populating a DataFrame column with empty or null values, it is recommended to use lit(None). Using literals such as
lit('') as a placeholder for absent values can lead to data misinterpretation and inconsistencies.
The usage of lit(None) ensures clarity and consistency in the codebase, making it explicit that the column is intentionally populated
with null values. Using lit(None) also preserves the ability to use functions such as isnull or isnotnull to
check for null values in the DataFrame.
