Skip to content

Commit

Permalink
Fix backward-compatibility (Spark 2.x) issue introduced with #24:
Browse files Browse the repository at this point in the history
Ignore errors by calling SparkSession.getActiveSession()
(available since Spark 3.0.0) and fall-back to use the
Python logging module
  • Loading branch information
sebastian-nagel committed Apr 21, 2022
1 parent ff8c744 commit 66dba9a
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion sparkcc.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,10 @@ def init_accumulators(self, session):
def get_logger(self, session=None):
"""Get logger from SparkSession or (if None) from logging module"""
if not session:
session = SparkSession.getActiveSession()
try:
session = SparkSession.getActiveSession()
except AttributeError:
pass # method available since Spark 3.0.0
if session:
return session._jvm.org.apache.log4j.LogManager \
.getLogger(self.name)
Expand Down

0 comments on commit 66dba9a

Please sign in to comment.