Skip to content
This repository was archived by the owner on Nov 28, 2025. It is now read-only.

Prepare spark-tensorflow-distributor 0.1.0 release #163

Open
mengxr wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Prepare spark-tensorflow-distributor 0.1.0 release #163

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
388be86
prepare 0.1.0 release
mengxr Jun 22, 2020
9199fe9
use Spark 3.0
mengxr Jun 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions spark/spark-tensorflow-distributor/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,7 @@ RUN apt-get update && \
update-alternatives --install /usr/local/bin/python python /usr/bin/python${PYTHON_INSTALL_VERSION} 1

# Install Spark and update env variables
# Modify this when the next Spark 3 release includes the allGather barrier mode API
RUN pip install \
https://ml-team-public-read.s3-us-west-2.amazonaws.com/pyspark-3.1.0.dev0-60dd1a690fed62b1d6442cdc8cf3f89ef4304d5a.tar.gz \
--force-reinstall --upgrade
RUN pip install pyspark==3.0.0

# Set SPARK_HOME so that tests can easily find the scripts in $SPARK_HOME/sbin
ENV SPARK_HOME /usr/local/lib/python${PYTHON_INSTALL_VERSION}/dist-packages/pyspark
Expand Down
2 changes: 1 addition & 1 deletion spark/spark-tensorflow-distributor/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

setuptools.setup(
name="spark_tensorflow_distributor",
version="0.0.3",
version="0.1.0",
author="sarthfrey",
author_email="sarth.frey@gmail.com",
description="This package helps users do distributed training with TensorFlow on their Spark clusters.",
Expand Down