Skip to content

Commit

Permalink
[SC-37900] [Delta] Updated Delta dependencies to Spark 3.0 final
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

- as the title says

## How was this patch tested?
all tests

Author: Tathagata Das <[email protected]>

#10658 is resolved by tdas/SC-37900.

GitOrigin-RevId: 8a8994dfd7979185294697bfdd65642a7a230653
  • Loading branch information
tdas committed Jun 16, 2020
1 parent d4407d5 commit a0db1e7
Show file tree
Hide file tree
Showing 4 changed files with 3 additions and 6 deletions.
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
pyenv install 3.7.4
pyenv global system 3.7.4
pipenv --python 3.7 install
pipenv run pip install https://docs.delta.io/spark3artifacts/snapshot-fa608b94/distributions/pyspark-3.0.0.tar.gz
pipenv run pip install pyspark>=3.0.0
pipenv run pip install flake8==3.5.0 pypandoc==1.3.3
- run:
name: Run Scala/Java and Python tests
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM python:3.7.3-stretch

RUN apt-get update && apt-get -y install openjdk-8-jdk

RUN pip install https://docs.delta.io/spark3artifacts/snapshot-fa608b94/distributions/pyspark-3.0.0.tar.gz
RUN pip install pyspark>=3.0.0

COPY . /usr/src/delta

Expand Down
4 changes: 1 addition & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ organization := "io.delta"

scalaVersion := "2.12.10"

sparkVersion := "3.0.1-SNAPSHOT"
sparkVersion := "3.0.0"

libraryDependencies ++= Seq(
// Adding test classifier seems to break transitive resolution of the core dependencies
Expand All @@ -43,8 +43,6 @@ libraryDependencies ++= Seq(
compilerPlugin("com.typesafe.genjavadoc" %% "genjavadoc-plugin" % "0.16" cross CrossVersion.full)
)

resolvers += "Temporary Staging of Spark 3.0" at "https://docs.delta.io/spark3artifacts/snapshot-fa608b94/maven/"

antlr4Settings

antlr4Version in Antlr4 := "4.7"
Expand Down
1 change: 0 additions & 1 deletion build/sbt-config/repositories
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,3 @@
bintray-typesafe-sbt-plugin-releases: https://dl.bintray.com/typesafe/sbt-plugins/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
bintray-spark-packages: https://dl.bintray.com/spark-packages/maven/
typesafe-releases: http://repo.typesafe.com/typesafe/releases/
temporary-staging-for-spark-3.0: https://docs.delta.io/spark3artifacts/snapshot-fa608b94/maven/

0 comments on commit a0db1e7

Please sign in to comment.