From ec026972e74f185ea98768f94fb52b990326aed2 Mon Sep 17 00:00:00 2001 From: lergor Date: Mon, 18 Mar 2019 00:50:13 +0300 Subject: [PATCH 1/4] add readme --- torrent/Readme.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 torrent/Readme.md diff --git a/torrent/Readme.md b/torrent/Readme.md new file mode 100644 index 0000000..9fcc36a --- /dev/null +++ b/torrent/Readme.md @@ -0,0 +1,45 @@ +# Torrent + +### Description + +This repository contains simple torrent with tracker and client applications. + +### How to run + +In order to run this app, follow these steps: + +* clone this repository +* checkout on branch *torrent* +* run `gradle trackerJar` and `gradle clientJar` to build jar for TrackerApp and ClientApp respectively +* go to the `build/libs` directory +* run ```java -jar trackerApp-jar-1.0-SNAPSHOT.jar``` and ```java -jar clientApp-jar-1.0-SNAPSHOT.jar``` to run Tracker or Client respectively + +### Usage + +Tracker:
+``` +exit - shutdown the tracker app +``` + +Client:
+``` +help - print usage +list - list available files on the tracker +sources - list sources for file with the specified id +upload - add file to the tracker +download - download file with the specified id +exit - shutdown the client app +``` + +### File hierarchy +``` +torrent/ - directory for downloads + .metainfo/ - directory for torrent metafiles + client/ + parts/ + [id]/[part] - directories named as id for parts of file + local_files_manager_file - stored client state with info about local files + tracker/ + tracker_state_file - stored tracker state with info about available files + +``` From 83e3728f0e565eca74e6ffa512e739c47cd1b9e9 Mon Sep 17 00:00:00 2001 From: lergor Date: Mon, 18 Mar 2019 01:16:23 +0300 Subject: [PATCH 2/4] add readme --- torrent/Readme.md | 143 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 torrent/Readme.md diff --git a/torrent/Readme.md b/torrent/Readme.md new file mode 100644 index 0000000..fb8b9ef --- /dev/null +++ b/torrent/Readme.md @@ -0,0 +1,143 @@ +# Torrent +* На трекере хранится список файлов и информация об активных пользователях, у которых есть те или иные файлы (возможно не целиком). +* С помощью клиентского приложения можно просматривать список файлов на трекере, а также добавлять новые и выбирать файлы из списка для скачивания. +* Файлы условно разбиваются на последовательные блоки бинарных данных константного размера (например 10M). Последний блок может иметь меньший размер. Блоки нумеруются с нуля. +* Клиент при подключении отправляет на трекер список раздаваемых им файлов. +* При скачивании файла клиент получает у трекера информацию о клиентах, раздающих файл (сидах), и далее общается с ними напрямую. +* У отдельного сида можно узнать о том, какие полные части у него есть, а также скачать их. +* После скачивания отдельных блоков некоторого файла клиент становится сидом. +--- +# Torrent-tracker +* Хранит мета-информацию о клиентах и раздаваемых файлах + +* **Порт сервера: 8081** + +* Запросы: + * list — список раздаваемых файлов + * upload — публикация нового файла + * sources — список клиентов, владеющих определенным файлов целиком или некоторыми его частями + * update — загрузка клиентом данных о раздаваемых файлах + +--- +### List +Формат запроса: + <1: Byte> +Формат ответа: + ( )*, + count — количество файлов + id — идентификатор файла + name — название файла + size — размер файла + +--- +### Upload +Формат запроса: + <2: Byte> , + name — название файла + size — размер файла +Формат ответа: + , + id — идентификатор файла + +**Примечание:**
+* Если клиент А и клиент Б решили опубликовать файл abc.txt, то это будут **разные** файлы, иными словами каждый запрос на публикацию файла возвращает **новый** id +--- +### Sources + +Формат запроса: + <3: Byte> , + id — идентификатор файла +Формат ответа: + ( )*, + size — количество клиентов, раздающих файл + ip — ip клиента, +clientPort — порт клиента + +--- + +### Update + +Формат запроса: + <4: Byte> ()*, + clientPort — порт клиента, + count — количество раздаваемых файлов, + id — идентификатор файла +Формат ответа: + , + status — True, если информация успешно обновлена + +**Примечание:**
+* Клиент обязан исполнять данный запрос каждые 5 минут, иначе сервер считает, что клиент ушел с раздачи +--- +# Torrent-client +* Порт клиента указывается при запуске и передается на трекер в рамках запроса update +* Каждый файл раздается по частям, размер части — константа на всё приложение +* Клиент хранит и раздает эти самые части +* Запросы: + * stat — доступные для раздачи части определенного файла + * get — скачивание части определенного файла + +--- +### Stat +Формат запроса: + <1: Byte> , + id — идентификатор файла +Формат ответа: + ()*, + count — количество доступных частей + part — номер части + +**Примечание:**
+* Часть считается доступной для раздачи, если она хранится на клиенте целиком +--- +### Get +Формат запроса: + <2: Byte> + id — идентификатор файла, + part — номер части +Формат ответа: + + , + content — содержимое части +--- + +# Запуск + +Для запуска приложения нужно выполнить следующие шаги: + +* склонировать [репозиторий](https://github.com/lergor/NetworksLab2019HSE.git) +* переключиться на нужную ветку: `git co torrent` +* запустить команды `gradle trackerJar` и `gradle clientJar` для сборки приложений трекера и клиента -- TrackerApp и ClientApp соответственно +* перейти в директорию `build/libs` +* запустить трекер: ```java -jar trackerApp-jar-1.0-SNAPSHOT.jar``` +* запустить клиент: ```java -jar clientApp-jar-1.0-SNAPSHOT.jar``` + +## Использование + +**Трекер:**
+``` +exit -- завершить работу приложения +``` + +**Клиент:**
+``` +help - напечатать список доступных комнад +list - получить список раздаваемых файлов +sources - получить список клиентов, владеющих определенным файлов целиком или некоторыми его частями +upload - добавить файл на раздачу +download - скачать файл с указанным `id` +exit - завершить работу приложения +``` + +## File hierarchy +``` +torrent/ - directory for downloads + .metainfo/ - directory for torrent metafiles + client/ + parts/ + [id]/[part] - directories named as id for parts of file + local_files_manager_file - stored client state with info about local files + tracker/ + tracker_state_file - stored tracker state with info about available files + +``` \ No newline at end of file From ff822c121ccde86f405baa6816036b6f8667464c Mon Sep 17 00:00:00 2001 From: lergor Date: Mon, 18 Mar 2019 01:16:48 +0300 Subject: [PATCH 3/4] torrent --- torrent/build.gradle | 37 ++++ .../gradle/wrapper/gradle-wrapper.properties | 6 + torrent/gradlew | 172 ++++++++++++++++++ torrent/gradlew.bat | 84 +++++++++ torrent/settings.gradle | 3 + .../java/ru/ifmo/torrent/client/Client.java | 126 +++++++++++++ .../ru/ifmo/torrent/client/ClientApp.java | 133 ++++++++++++++ .../ru/ifmo/torrent/client/ClientConfig.java | 30 +++ .../ifmo/torrent/client/leech/Downloader.java | 142 +++++++++++++++ .../ifmo/torrent/client/leech/FilePart.java | 35 ++++ .../ru/ifmo/torrent/client/leech/Leecher.java | 53 ++++++ .../torrent/client/seed/LeechHandler.java | 73 ++++++++ .../ru/ifmo/torrent/client/seed/Seeder.java | 48 +++++ .../client/storage/LocalFileReference.java | 103 +++++++++++ .../client/storage/LocalFilesManager.java | 109 +++++++++++ .../torrent/client/storage/PartsManager.java | 81 +++++++++ .../client/storage/SourcesUpdater.java | 49 +++++ .../ifmo/torrent/messages/NetworkMessage.java | 12 ++ .../ru/ifmo/torrent/messages/Request.java | 18 ++ .../ru/ifmo/torrent/messages/Response.java | 3 + .../messages/client_tracker/Marker.java | 8 + .../client_tracker/requests/ListRequest.java | 33 ++++ .../requests/SourcesRequest.java | 47 +++++ .../requests/UpdateRequest.java | 64 +++++++ .../requests/UploadRequest.java | 57 ++++++ .../client_tracker/response/ListResponse.java | 49 +++++ .../response/SourcesResponse.java | 57 ++++++ .../response/UpdateResponse.java | 33 ++++ .../response/UploadResponse.java | 34 ++++ .../torrent/messages/seed_peer/Marker.java | 6 + .../seed_peer/requests/GetRequest.java | 56 ++++++ .../seed_peer/requests/StatRequest.java | 47 +++++ .../seed_peer/response/GetResponse.java | 40 ++++ .../seed_peer/response/StatResponse.java | 41 +++++ .../ifmo/torrent/tracker/ClientHandler.java | 85 +++++++++ .../java/ru/ifmo/torrent/tracker/Tracker.java | 57 ++++++ .../ru/ifmo/torrent/tracker/TrackerApp.java | 31 ++++ .../ifmo/torrent/tracker/TrackerConfig.java | 27 +++ .../ifmo/torrent/tracker/state/FileInfo.java | 59 ++++++ .../ifmo/torrent/tracker/state/SeedInfo.java | 48 +++++ .../torrent/tracker/state/TimedSeedInfo.java | 38 ++++ .../torrent/tracker/state/TrackerState.java | 109 +++++++++++ .../java/ru/ifmo/torrent/util/Config.java | 14 ++ .../ru/ifmo/torrent/util/StoredState.java | 8 + .../ifmo/torrent/util/TorrentException.java | 32 ++++ .../java/ru/ifmo/torrent/TorrentTest.java | 118 ++++++++++++ .../client/storage/LocalFilesManagerTest.java | 61 +++++++ .../client/storage/PartsManagerTest.java | 59 ++++++ .../client_tracker/RequestsTests.java | 82 +++++++++ .../client_tracker/ResponseTests.java | 89 +++++++++ .../messages/seed_peer/RequestTests.java | 51 ++++++ .../messages/seed_peer/ResponseTests.java | 53 ++++++ .../tracker/state/TrackerStateTest.java | 67 +++++++ 53 files changed, 2977 insertions(+) create mode 100644 torrent/build.gradle create mode 100644 torrent/gradle/wrapper/gradle-wrapper.properties create mode 100755 torrent/gradlew create mode 100644 torrent/gradlew.bat create mode 100644 torrent/settings.gradle create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/Client.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/ClientApp.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/ClientConfig.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/leech/Downloader.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/leech/FilePart.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/leech/Leecher.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/seed/LeechHandler.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/seed/Seeder.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFileReference.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFilesManager.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/storage/PartsManager.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/client/storage/SourcesUpdater.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/NetworkMessage.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/Request.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/Response.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/Marker.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/ListRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/SourcesRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UpdateRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UploadRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/ListResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/SourcesResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UpdateResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UploadResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/Marker.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/GetRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/StatRequest.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/GetResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/StatResponse.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/ClientHandler.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/Tracker.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerApp.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerConfig.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/state/FileInfo.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/state/SeedInfo.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/state/TimedSeedInfo.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/tracker/state/TrackerState.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/util/Config.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/util/StoredState.java create mode 100644 torrent/src/main/java/ru/ifmo/torrent/util/TorrentException.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/TorrentTest.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/client/storage/LocalFilesManagerTest.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/client/storage/PartsManagerTest.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/RequestsTests.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/ResponseTests.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/RequestTests.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/ResponseTests.java create mode 100644 torrent/src/test/java/ru/ifmo/torrent/tracker/state/TrackerStateTest.java diff --git a/torrent/build.gradle b/torrent/build.gradle new file mode 100644 index 0000000..dd80738 --- /dev/null +++ b/torrent/build.gradle @@ -0,0 +1,37 @@ +plugins { + id 'java' + id 'application' +} + +group 'ru.ifmo' +version '1.0-SNAPSHOT' + +sourceCompatibility = 1.8 + +repositories { + mavenCentral() +} + +task trackerJar(type: Jar) { + manifest { + attributes 'Main-Class': 'ru.ifmo.torrent.tracker.TrackerApp' + } + baseName = 'trackerApp-jar' + from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } } + with jar +} + +task clientJar(type: Jar) { + manifest { + attributes 'Main-Class': 'ru.ifmo.torrent.client.ClientApp' + } + baseName = 'clientApp-jar' + from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } } + with jar +} + +dependencies { + testCompile group: 'junit', name: 'junit', version: '4.12' + testCompile group: 'org.assertj', name: 'assertj-core', version: '3.9.0' + compile group: 'org.apache.commons', name: 'commons-io', version: '1.3.2' +} diff --git a/torrent/gradle/wrapper/gradle-wrapper.properties b/torrent/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000..9b5b666 --- /dev/null +++ b/torrent/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,6 @@ +#Mon Dec 17 05:46:23 MSK 2018 +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-4.0-all.zip diff --git a/torrent/gradlew b/torrent/gradlew new file mode 100755 index 0000000..cccdd3d --- /dev/null +++ b/torrent/gradlew @@ -0,0 +1,172 @@ +#!/usr/bin/env sh + +############################################################################## +## +## Gradle start up script for UN*X +## +############################################################################## + +# Attempt to set APP_HOME +# Resolve links: $0 may be a link +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS="" + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn () { + echo "$*" +} + +die () { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin, switch paths to Windows format before running java +if $cygwin ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=$((i+1)) + done + case $i in + (0) set -- ;; + (1) set -- "$args0" ;; + (2) set -- "$args0" "$args1" ;; + (3) set -- "$args0" "$args1" "$args2" ;; + (4) set -- "$args0" "$args1" "$args2" "$args3" ;; + (5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + (6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + (7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + (8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + (9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Escape application args +save () { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=$(save "$@") + +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" + +# by default we should be in the correct project dir, but when run from Finder on Mac, the cwd is wrong +if [ "$(uname)" = "Darwin" ] && [ "$HOME" = "$PWD" ]; then + cd "$(dirname "$0")" +fi + +exec "$JAVACMD" "$@" diff --git a/torrent/gradlew.bat b/torrent/gradlew.bat new file mode 100644 index 0000000..e95643d --- /dev/null +++ b/torrent/gradlew.bat @@ -0,0 +1,84 @@ +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS= + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto init + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto init + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:init +@rem Get command-line arguments, handling Windows variants + +if not "%OS%" == "Windows_NT" goto win9xME_args + +:win9xME_args +@rem Slurp the command line arguments. +set CMD_LINE_ARGS= +set _SKIP=2 + +:win9xME_args_slurp +if "x%~1" == "x" goto execute + +set CMD_LINE_ARGS=%* + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %CMD_LINE_ARGS% + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/torrent/settings.gradle b/torrent/settings.gradle new file mode 100644 index 0000000..e122bba --- /dev/null +++ b/torrent/settings.gradle @@ -0,0 +1,3 @@ +rootProject.name = 'torrent' +include 'client' + diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/Client.java b/torrent/src/main/java/ru/ifmo/torrent/client/Client.java new file mode 100644 index 0000000..0270195 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/Client.java @@ -0,0 +1,126 @@ +package ru.ifmo.torrent.client; + +import ru.ifmo.torrent.client.leech.Downloader; +import ru.ifmo.torrent.client.seed.Seeder; +import ru.ifmo.torrent.client.storage.*; +import ru.ifmo.torrent.messages.client_tracker.requests.*; +import ru.ifmo.torrent.messages.client_tracker.response.*; +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.tracker.state.FileInfo; +import ru.ifmo.torrent.tracker.state.SeedInfo; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.net.InetAddress; +import java.net.Socket; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; +import java.util.Optional; + +public class Client implements AutoCloseable { + + private InetAddress inetAddress; + private LocalFilesManager localFilesManager; + private Downloader downloader; + private Seeder seeder; + private final SourcesUpdater sourcesUpdater; + + public Client(InetAddress inetAddress, short port, Path metaDir, Path downloadDir) throws IOException, TorrentException { + this.inetAddress = inetAddress; + localFilesManager = new LocalFilesManager(downloadDir, metaDir.resolve(ClientConfig.LOCAL_FILES_FILE), metaDir.resolve(ClientConfig.PARTS_STORAGE)); + sourcesUpdater = new SourcesUpdater(this, localFilesManager, port); + + this.downloader = new Downloader(localFilesManager, this); + Thread downloaderTread = new Thread(downloader); + downloaderTread.start(); + + this.seeder = new Seeder(port, localFilesManager); + Thread seedTread = new Thread(seeder); + seedTread.start(); + + } + + public T sendRequest(Request request) throws IOException, TorrentException { + try (Socket clientSocket = new Socket(inetAddress, ClientConfig.TRACKER_PORT)) { + DataInputStream in = new DataInputStream(clientSocket.getInputStream()); + DataOutputStream out = new DataOutputStream(clientSocket.getOutputStream()); + + T response; + try { + request.write(out); + out.flush(); + response = request.getEmptyResponse(); + response.read(in); + } catch (IOException e) { + throw new TorrentException("cannot send request " + request.getClass().getSimpleName(), e); + } + return response; + } + } + + @Override + public void close() throws TorrentException { + try { + seeder.close(); + } finally { + downloader.close(); + sourcesUpdater.close(); + localFilesManager.storeToFile(); + } + } + + public List getAvailableFiles() throws IOException, TorrentException { + ListResponse response = sendRequest(new ListRequest()); + return response.getFiles(); + } + + public int uploadFile(Path file) throws IOException, TorrentException { + if (Files.notExists(file)) { + throw new TorrentException("file '" + file + "' does not exists"); + } + UploadRequest request = new UploadRequest(file); + UploadResponse response = sendRequest(request); + localFilesManager.addFileToStorageAsParts(response.getFileId(), file); + localFilesManager.addLocalFile(file.getFileName().toString(), response.getFileId(), request.getFileSize()); + return response.getFileId(); + } + + public List getFileSources(int fileId) throws TorrentException { + SourcesResponse response; + try { + response = sendRequest(new SourcesRequest(fileId)); + } catch (IOException e) { + return new ArrayList<>(); + } catch (TorrentException e) { + throw new TorrentException(e.getMessage(), e.getException()); + } + return response.getClients(); + } + + public boolean downloadFile(int fileId) throws IOException, TorrentException { + if (localFilesManager.getPartsManager().fileIsPresent(fileId)) { + throw new IllegalArgumentException("file with id " + fileId + " already added as local file"); + } + + Optional fileInfo = getAvailableFiles().stream() + .filter(f -> f.getId() == fileId) + .findFirst(); + + if(!fileInfo.isPresent()) { + throw new IllegalArgumentException("File with id " + fileId + " does not exist!"); + } + FileInfo file = fileInfo.get(); + localFilesManager.addNotDownloadedFile(file.getName(), file.getId(), file.getSize()); + return true; + } + + List getLocalFiles() { + return localFilesManager.getFiles(); + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/ClientApp.java b/torrent/src/main/java/ru/ifmo/torrent/client/ClientApp.java new file mode 100644 index 0000000..34d3ad5 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/ClientApp.java @@ -0,0 +1,133 @@ +package ru.ifmo.torrent.client; + +import ru.ifmo.torrent.client.storage.LocalFileReference; +import ru.ifmo.torrent.tracker.state.FileInfo; +import ru.ifmo.torrent.tracker.state.SeedInfo; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.net.InetAddress; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.InputMismatchException; +import java.util.List; +import java.util.Scanner; + +public class ClientApp { + + public static void main(String[] args) { + Scanner scanner = new Scanner(System.in); + + System.out.print("enter client port: "); + Short port = getPort(args, scanner); + + try (Client client = new Client(InetAddress.getLocalHost(), port, ClientConfig.getMetaDir(), ClientConfig.TORRENT_DIR)) { + System.out.printf("client started at getPort %d%n", port); + printUsage(); + main_while: + while (scanner.hasNext()) { + String command = scanner.next(); + switch (command) { + case Command.EXIT: { + System.out.println("shutting down client"); + break main_while; + } + case Command.LIST: { + List files = client.getAvailableFiles(); + System.out.printf("files count: %d%n", files.size()); + for (FileInfo f : files) { + System.out.printf("\t%s\tid: %d, getSize: %d bytes%n", + f.getName(), + f.getId(), + f.getSize() + ); + } + break; + } + case Command.UPLOAD: { + String path = scanner.next(); + Path file = Paths.get(path); + int fileId = client.uploadFile(file); + System.out.printf("file added with id: %d%n", fileId); + break; + } + case Command.SOURCES: { + int fileId = scanner.nextInt(); + List sources = client.getFileSources(fileId); + System.out.printf("sources count: %d%n", sources.size()); + for (SeedInfo s : sources) { + System.out.printf("\taddress: %s, getPort: %d%n", + s.getInetAddress(), + s.getPort() + ); + } + break; + } + case Command.DOWNLOAD: { + int fileId = scanner.nextInt(); + if(client.downloadFile(fileId)) { + System.out.println("file with id " + fileId + " downloaded"); + } + break; + } + case Command.STATS: { + List files = client.getLocalFiles(); + System.out.printf("local files count: %d%n", files.size()); + for (LocalFileReference f : files) { + System.out.printf( + "\tfile: %s, id: %d, parts: %d/%d%n", + f.getName(), + f.getFileId(), + f.getReadyParts().size(), + f.getNumberOfParts() + ); + } + break; + } + case Command.HELP : { + printUsage(); + break; + } + default: { + System.out.printf("client: unknown command: %s%n", command); + break; + } + } + } + } catch (TorrentException e) { + System.out.println(e.getMassage()); + if (e.getException() != null) e.getException().printStackTrace(); + } catch (IOException e) { + e.printStackTrace(); + } + } + + private static Short getPort(String[] args, Scanner scanner) { + if (args.length != 0) { + return Short.parseShort(args[0]); + } + return scanner.nextShort(); + } + + private static void printUsage() { + String sep = System.lineSeparator(); + System.out.println("available commands:" + sep + + Command.HELP + " - print this message" + sep + + Command.LIST + " - list available files on the tracker" + sep + + Command.SOURCES + " - list seeds for file with the specified id" + sep + + Command.UPLOAD + " - add file to the tracker" + sep + + Command.DOWNLOAD + " - download file with the specified id" + sep + + Command.EXIT + " - shutdown the client app" + sep + ); + } + + private static class Command { + static final String HELP = "help"; + static final String LIST = "list"; + static final String UPLOAD = "upload"; + static final String SOURCES = "sources"; + static final String EXIT = "exit"; + static final String DOWNLOAD = "download"; + static final String STATS = "stats"; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/ClientConfig.java b/torrent/src/main/java/ru/ifmo/torrent/client/ClientConfig.java new file mode 100644 index 0000000..9a32fd0 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/ClientConfig.java @@ -0,0 +1,30 @@ +package ru.ifmo.torrent.client; + +import ru.ifmo.torrent.util.Config; + +import java.nio.file.Files; +import java.nio.file.Path; + +public class ClientConfig extends Config { + + public static final int FILE_PART_SIZE = 1024 * 1024 * 10; + public static final int UPDATE_RATE_SEC = 7; + public static final int DOWNLOAD_RATE_SEC = 2; + public static final int TRACKER_PORT = 8081; + public static final int SEED_THREADS_COUNT = 4; + public static final int DOWNLOADS_LIMIT = 5; + + public static final String PARTS_STORAGE = "parts"; + public static final String LOCAL_FILES_FILE = "local_files_manager_file"; + + private ClientConfig() {} + + public static Path getMetaDir() { + Path storage = CLIENT_STORAGE; + if(Files.notExists(storage)) { + storage.toFile().mkdirs(); + } + return storage; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/leech/Downloader.java b/torrent/src/main/java/ru/ifmo/torrent/client/leech/Downloader.java new file mode 100644 index 0000000..8b54d56 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/leech/Downloader.java @@ -0,0 +1,142 @@ +package ru.ifmo.torrent.client.leech; + +import ru.ifmo.torrent.client.Client; +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.client.storage.LocalFilesManager; +import ru.ifmo.torrent.client.storage.PartsManager; +import ru.ifmo.torrent.tracker.state.SeedInfo; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.*; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.RejectedExecutionException; +import java.util.stream.Collectors; + +public class Downloader implements Runnable, AutoCloseable { + + private static final int DOWNLOADS_LIMIT = ClientConfig.DOWNLOADS_LIMIT; + + private final ExecutorService pool = Executors.newFixedThreadPool(DOWNLOADS_LIMIT); + private final LocalFilesManager filesManager; + private final Client client; + private final Set downloadingParts; + private boolean shouldExit = false; + + public Downloader(LocalFilesManager filesManager, Client client) { + this.filesManager = filesManager; + downloadingParts = new HashSet<>(); + this.client = client; + } + + @Override + public void close() { + shouldExit = true; + pool.shutdown(); + } + + public void updateDownloads() { + Set partsToDownload = filesManager.getFiles().stream().flatMap( + f -> f.getMissingParts().stream() + .map(p -> new FilePart(f.getFileId(), p)) + .filter(p -> !downloadingParts.contains(p) + ) + ).collect(Collectors.toSet()); + + Set fileIds = partsToDownload.stream().map(FilePart::getFileId).collect(Collectors.toSet()); + + Map> sourcesForFile = getSourcesForFiles(fileIds); + partsToDownload.stream().filter(p -> !sourcesForFile.get(p.getFileId()).isEmpty()) + .limit(DOWNLOADS_LIMIT - downloadingParts.size()) + .forEach(p -> downloadPart(p, sourcesForFile.get(p.getFileId()))); + } + + private Map> getSourcesForFiles(Set fileIds) { + Map> sources = new HashMap<>(); + for (Integer fileId : fileIds) { + try { + sources.put(fileId, client.getFileSources(fileId)); + } catch (TorrentException e) { + e.printStackTrace(); + } + } + return sources; + } + + private void downloadPart(FilePart part, List sources) { + if (!sources.isEmpty()) { + try { + pool.submit(new DownloadTask(part, sources)); + } catch (RejectedExecutionException e) { + shouldExit = true; + } + } + } + + @Override + public void run() { + while (!shouldExit) { + try { + Thread.sleep(ClientConfig.DOWNLOAD_RATE_SEC * 1000); + } catch (InterruptedException e) { + break; + } + updateDownloads(); + } + } + + private class DownloadTask implements Runnable { + + private final FilePart part; + private final List sources; + private PartsManager partsManager = filesManager.getPartsManager(); + + DownloadTask(FilePart part, List sources) { + this.part = part; + this.sources = sources; + downloadingParts.add(part); + } + + @Override + public void run() { + try { + Optional maybeSource = getSource(); + if (!maybeSource.isPresent()) return; + + SeedInfo source = maybeSource.get(); + Leecher leecher = new Leecher(source.getPort(), source.getInetAddress()); + + byte[] content = leecher.getPartContent(part.getFileId(), part.getPartNum()); + + try (OutputStream out = partsManager.getForWriting(part.getFileId(), part.getPartNum())) { + out.write(content); + out.flush(); + } + + filesManager.addReadyPartOfFile(part.getFileId(), part.getPartNum()); + downloadingParts.remove(part); + + } catch (IOException e) { + System.err.printf("error while downloading file with id %d%n", part.getFileId()); + e.printStackTrace(); + } catch (TorrentException e) { + System.err.println(e.getMassage()); + e.getException().printStackTrace(); + } + } + + private Optional getSource() throws TorrentException { + for (SeedInfo s : sources) { + Leecher leecher = new Leecher(s.getPort(), s.getInetAddress()); + List availableParts = leecher.getAvailableParts(part.getFileId()); + if (availableParts.contains(part.getPartNum())) { + return Optional.of(s); + } + } + return Optional.empty(); + } + } + +} \ No newline at end of file diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/leech/FilePart.java b/torrent/src/main/java/ru/ifmo/torrent/client/leech/FilePart.java new file mode 100644 index 0000000..6929f20 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/leech/FilePart.java @@ -0,0 +1,35 @@ +package ru.ifmo.torrent.client.leech; + +import java.util.Objects; + +public class FilePart { + private final int fileId; + private final int num; + + public FilePart(int fileId, int partNum) { + this.fileId = fileId; + this.num = partNum; + } + + public int getFileId() { + return fileId; + } + + public int getPartNum() { + return num; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + FilePart filePart = (FilePart) o; + return fileId == filePart.fileId && num == filePart.num; + } + + @Override + public int hashCode() { + return Objects.hash(fileId, num); + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/leech/Leecher.java b/torrent/src/main/java/ru/ifmo/torrent/client/leech/Leecher.java new file mode 100644 index 0000000..2fb2183 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/leech/Leecher.java @@ -0,0 +1,53 @@ +package ru.ifmo.torrent.client.leech; + +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.messages.seed_peer.requests.*; +import ru.ifmo.torrent.messages.seed_peer.response.*; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.net.InetAddress; +import java.net.Socket; +import java.util.List; + +public class Leecher { + + private short port; + private final InetAddress address; + + + public Leecher(short port, InetAddress address) { + this.port = port; + this.address = address; + } + + public Response sendRequest(Request request) throws TorrentException { + try(Socket socket = new Socket(address, port)) { + DataInputStream in = new DataInputStream(socket.getInputStream()); + DataOutputStream out = new DataOutputStream(socket.getOutputStream()); + + request.write(out); + out.flush(); + + Response response = request.getEmptyResponse(); + response.read(in); + return response; + } catch (IOException e) { + throw new TorrentException("leecher: cannot send request of type " + request.marker()); + } + } + + public List getAvailableParts(int fileId) throws TorrentException { + StatResponse response = (StatResponse) sendRequest(new StatRequest(fileId)); + return response.getAvailableParts(); + } + + public byte[] getPartContent(int fileId, int part) throws TorrentException { + GetRequest request = new GetRequest(fileId, part); + GetResponse response = (GetResponse) sendRequest(request); + return response.getContent(); + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/seed/LeechHandler.java b/torrent/src/main/java/ru/ifmo/torrent/client/seed/LeechHandler.java new file mode 100644 index 0000000..420d264 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/seed/LeechHandler.java @@ -0,0 +1,73 @@ +package ru.ifmo.torrent.client.seed; + +import ru.ifmo.torrent.client.storage.LocalFileReference; +import ru.ifmo.torrent.client.storage.LocalFilesManager; +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.seed_peer.Marker; +import ru.ifmo.torrent.messages.seed_peer.requests.*; +import ru.ifmo.torrent.messages.seed_peer.response.*; +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.net.Socket; +import java.util.List; + +class LeechHandler implements Runnable { + + private final Socket leechSocket; + private final LocalFilesManager filesManager; + + LeechHandler(Socket leecher, LocalFilesManager localFilesManager) { + this.leechSocket = leecher; + this.filesManager = localFilesManager; + } + + @Override + public void run() { + try (Socket socket = leechSocket) { + DataInputStream in = new DataInputStream(socket.getInputStream()); + DataOutputStream out = new DataOutputStream(socket.getOutputStream()); + + Response response = null; + int marker = in.readByte(); + switch (marker) { + case Marker.GET: { + GetRequest request = (GetRequest) Request.readFromDataInputStream(in, GetRequest.class); + InputStream is = getPartForDownloading(request.getFileId(), request.getPart()); + LocalFileReference reference = filesManager.getFileReference(request.getFileId()); + response = new GetResponse(is, reference.getBlockSizeForPart(request.getPart())); + is.close(); + break; + } + case Marker.STAT: { + StatRequest request = (StatRequest) StatRequest.readFromDataInputStream(in, StatRequest.class); + response = new StatResponse(getParts(request.getFileId())); + break; + } + default: + break; + } + + if (response != null) { + response.write(out); + out.flush(); + } + + } catch (IOException | IllegalAccessException | InstantiationException e) { + System.err.printf("error while service leech %s %d%n", leechSocket.getInetAddress(), leechSocket.getPort()); + } + + } + + private InputStream getPartForDownloading(int fileId, int filePart) throws IOException { + return filesManager.getPartsManager().getForReading(fileId, filePart); + } + + private List getParts(int fileId) { + LocalFileReference file = filesManager.getFileReference(fileId); + return file.getReadyParts(); + } +} \ No newline at end of file diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/seed/Seeder.java b/torrent/src/main/java/ru/ifmo/torrent/client/seed/Seeder.java new file mode 100644 index 0000000..b0c44aa --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/seed/Seeder.java @@ -0,0 +1,48 @@ +package ru.ifmo.torrent.client.seed; + +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.client.storage.LocalFilesManager; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.net.ServerSocket; +import java.net.Socket; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; + +public class Seeder implements Runnable, AutoCloseable { + + private final LocalFilesManager filesManager; + private final ServerSocket socket; + private final ExecutorService pool = Executors.newFixedThreadPool(ClientConfig.SEED_THREADS_COUNT); + + + public Seeder(short port, LocalFilesManager filesManager) throws IOException { + this.filesManager = filesManager; + socket = new ServerSocket(port); + } + + public void run() { + try (ServerSocket socket = this.socket) { + while (true) { + Socket leecherSocket = socket.accept(); + pool.submit(new LeechHandler(leecherSocket, filesManager)); + } + } catch (IOException e) { + if(!socket.isClosed()) { + throw new IllegalStateException("cannot open seed socket", e); + } + } + } + + @Override + public void close() throws TorrentException { + try { + socket.close(); + } catch (IOException e) { + throw new TorrentException("cannot close seed socket properly", e); + } finally { + pool.shutdown(); + } + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFileReference.java b/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFileReference.java new file mode 100644 index 0000000..99ee2f6 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFileReference.java @@ -0,0 +1,103 @@ +package ru.ifmo.torrent.client.storage; + +import ru.ifmo.torrent.client.ClientConfig; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.IntStream; + +public class LocalFileReference { + + private final int fileId; + private final long size; + private final int numberOfParts; + private final Set readyParts; + private final String name; + + private LocalFileReference(String name, int fileId, long size, int numberOfParts, Set readyParts) { + this.fileId = fileId; + this.name = name; + this.numberOfParts = numberOfParts; + this.readyParts = readyParts; + this.size = size; + } + + public static LocalFileReference createEmpty(String name, int fileId, long size, int numberOfParts) { + return new LocalFileReference(name, fileId, size, numberOfParts, new HashSet<>()); + } + + public static LocalFileReference createFull(String name, int fileId, long size, int numberOfParts) { + Set readyParts = IntStream.range(0, numberOfParts).boxed().collect(Collectors.toSet()); + return new LocalFileReference(name, fileId, size, numberOfParts, readyParts); + } + + public int getFileId() { + return fileId; + } + + public int getNumberOfParts() { + return numberOfParts; + } + + public List getReadyParts() { + return new ArrayList<>(readyParts); + } + + public String getName() { + return name; + } + + public long getSize() { + return size; + } + + public void addReadyPart(int part) { + if (part < numberOfParts) { + readyParts.add(part); + } + } + + public static LocalFileReference readFrom(DataInputStream in) throws IOException { + int id = in.readInt(); + long size = in.readLong(); + String name = in.readUTF(); + int numOfParts = in.readInt(); + int numOfReadyParts = in.readInt(); + Set readyParts = new HashSet<>(); + for (int i = 0; i < numOfReadyParts; i++) { + int part = in.readInt(); + readyParts.add(part); + } + return new LocalFileReference(name, id, size, numOfParts, readyParts); + } + + public void write(DataOutputStream out) throws IOException { + out.writeInt(fileId); + out.writeLong(size); + out.writeUTF(name); + out.writeInt(numberOfParts); + out.writeInt(readyParts.size()); + for (Integer part : readyParts) { + out.writeInt(part); + } + out.flush(); + } + + public List getMissingParts() { + return IntStream.range(0, numberOfParts).boxed() + .filter(i -> !readyParts.contains(i)) + .collect(Collectors.toList()); + } + + public int getBlockSizeForPart(int part) { + int fullPartSize = ClientConfig.FILE_PART_SIZE; + return (part + 1) < numberOfParts ? fullPartSize : (int) (size - (numberOfParts - 1) * fullPartSize); + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFilesManager.java b/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFilesManager.java new file mode 100644 index 0000000..4cfcd6d --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/storage/LocalFilesManager.java @@ -0,0 +1,109 @@ +package ru.ifmo.torrent.client.storage; + +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.util.StoredState; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.*; +import java.util.concurrent.ConcurrentHashMap; + +public class LocalFilesManager implements StoredState { + + private ConcurrentHashMap localFiles = new ConcurrentHashMap<>(); + private PartsManager partsManager; + private final Path metaFile; + private final Path downloadDir; + + public LocalFilesManager(Path downloadDir, Path metaFile, Path partsStorage) throws IOException, TorrentException { + this.metaFile = metaFile; + this.downloadDir = downloadDir; + if (Files.notExists(metaFile)) { + metaFile.getParent().toFile().mkdirs(); + Files.createFile(metaFile); + } + partsManager = new PartsManager(partsStorage); + restoreFromFile(); + } + + public void addLocalFile(String name, int fileId, long size) { + int partsNum = getPartsNumber(size); + localFiles.putIfAbsent(fileId, LocalFileReference.createFull(name, fileId, size, partsNum)); + } + + public void addNotDownloadedFile(String name, int fileId, long size) { + LocalFileReference reference = LocalFileReference.createEmpty(name, fileId, size, getPartsNumber(size)); + localFiles.put(fileId, reference); + } + + public void addReadyPartOfFile(int fileId, int part) throws IOException { + getOrThrow(fileId).addReadyPart(part); + if (getOrThrow(fileId).getMissingParts().isEmpty()) { + LocalFileReference reference = localFiles.get(fileId); + String fileName = reference.getName(); + long fileSize = reference.getSize(); + partsManager.mergeSplitted(fileId, fileSize, downloadDir.resolve(fileName)); + } + } + + private LocalFileReference getOrThrow(int fileId) { + return Objects.requireNonNull(localFiles.get(fileId), "no file with id " + fileId); + } + + public LocalFileReference getFileReference(int fileId) { + return getOrThrow(fileId); + } + + public List getFiles() { + return new ArrayList<>(localFiles.values()); + } + + private int getPartsNumber(long size) { + return (int) Math.ceil(size / (double) ClientConfig.FILE_PART_SIZE); + } + + @Override + public void storeToFile() throws TorrentException { + try (DataOutputStream out = new DataOutputStream(Files.newOutputStream(metaFile))) { + out.writeInt(localFiles.size()); + for (LocalFileReference file : localFiles.values()) { + file.write(out); + } + out.flush(); + } catch (IOException e) { + throw new TorrentException("cannot save local files manager state", e); + } + } + + @Override + public void restoreFromFile() throws TorrentException { + try { + if (Files.size(metaFile) == 0) return; + localFiles = new ConcurrentHashMap<>(); + try (DataInputStream in = new DataInputStream(Files.newInputStream(metaFile))) { + int numOfLocalFiles = in.readInt(); + for (int i = 0; i < numOfLocalFiles; i++) { + LocalFileReference file = LocalFileReference.readFrom(in); + localFiles.put(file.getFileId(), file); + } + } + } catch (IOException e) { + throw new TorrentException("cannot restore local files manager state", e); + } + } + + public PartsManager getPartsManager() { + return partsManager; + } + + public void addFileToStorageAsParts(int fileId, Path file) throws IOException { + long size = Files.size(file); + LocalFileReference reference = LocalFileReference.createEmpty(file.getFileName().toString(), fileId, size, getPartsNumber(size)); + partsManager.storeSplitted(reference, file); + + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/storage/PartsManager.java b/torrent/src/main/java/ru/ifmo/torrent/client/storage/PartsManager.java new file mode 100644 index 0000000..248dbd6 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/storage/PartsManager.java @@ -0,0 +1,81 @@ +package ru.ifmo.torrent.client.storage; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.io.RandomAccessFile; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.StandardOpenOption; +import java.util.Comparator; +import java.util.List; +import java.util.stream.Collectors; + +public class PartsManager { + + private final Path storage; + + public PartsManager(Path storage) { + this.storage = storage; + if (Files.notExists(storage)) { + storage.toFile().mkdirs(); + } + } + + public void storeSplitted(LocalFileReference reference, Path targetFile) throws IOException { + try (InputStream is = Files.newInputStream(targetFile)) { + for (int i = 0; i < reference.getNumberOfParts(); i++) { + int partSize = reference.getBlockSizeForPart(i); + byte[] buf = new byte[partSize]; + int totalReaded = 0; + try (OutputStream out = getForWriting(reference.getFileId(), i)) { + while (totalReaded != partSize) { + int readed = is.read(buf); + out.write(buf, totalReaded, readed); + totalReaded += readed; + } + } + } + } + } + + public void mergeSplitted(int fileId, long size, Path targetFile) throws IOException { + Path fileDir = storage.resolve(String.valueOf(fileId)); + List parts = Files.list(fileDir) + .sorted(Comparator.comparing(this::parsePartName)) + .collect(Collectors.toList()); + if (Files.notExists(targetFile)) { + Files.createDirectories(targetFile.getParent()); + try (RandomAccessFile randomAccessFile = new RandomAccessFile(targetFile.toFile(), "rw")) { + randomAccessFile.setLength(size); + } + } + OutputStream out = Files.newOutputStream(targetFile, StandardOpenOption.TRUNCATE_EXISTING); + for (Path p : parts) { + Files.copy(p, out); + } + } + + private int parsePartName(Path path) { + return Integer.parseInt(path.getFileName().toString()); + } + + public OutputStream getForWriting(int fileId, int part) throws IOException { + Path partFile = storage.resolve(String.valueOf(fileId)).resolve(String.valueOf(part)); + if (Files.notExists(partFile)) { + Files.createDirectories(partFile.getParent()); + Files.createFile(partFile); + } + return Files.newOutputStream(partFile, StandardOpenOption.TRUNCATE_EXISTING); + } + + public InputStream getForReading(int fileId, int part) throws IOException { + Path partFile = storage.resolve(String.valueOf(fileId)).resolve(String.valueOf(part)); + return Files.newInputStream(partFile); + } + + public boolean fileIsPresent(int fileId) { + return Files.exists(storage.resolve(String.valueOf(fileId))); + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/client/storage/SourcesUpdater.java b/torrent/src/main/java/ru/ifmo/torrent/client/storage/SourcesUpdater.java new file mode 100644 index 0000000..8650c82 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/client/storage/SourcesUpdater.java @@ -0,0 +1,49 @@ +package ru.ifmo.torrent.client.storage; + +import ru.ifmo.torrent.client.Client; +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.messages.client_tracker.requests.UpdateRequest; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.util.List; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; + +public class SourcesUpdater implements AutoCloseable { + + private final ScheduledExecutorService pool; + private final LocalFilesManager filesManager; + private final Client client; + private final short clientPort; + + public SourcesUpdater(Client client, LocalFilesManager filesManager, short clientPort) { + pool = Executors.newScheduledThreadPool(1); + pool.scheduleAtFixedRate(this::updateSources, 0, ClientConfig.UPDATE_RATE_SEC, TimeUnit.SECONDS); + this.filesManager = filesManager; + this.client = client; + this.clientPort = clientPort; + } + + private void updateSources() { + List fileIds = filesManager.getFiles().stream() + .filter(f -> !f.getReadyParts().isEmpty()) + .map(LocalFileReference::getFileId) + .collect(Collectors.toList()); + + try { + if(!fileIds.isEmpty()) { + client.sendRequest(new UpdateRequest(clientPort, fileIds)); + } + } catch (IOException | TorrentException ignored) { + } + } + + @Override + public void close() { + pool.shutdown(); + } + +} \ No newline at end of file diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/NetworkMessage.java b/torrent/src/main/java/ru/ifmo/torrent/messages/NetworkMessage.java new file mode 100644 index 0000000..4708d20 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/NetworkMessage.java @@ -0,0 +1,12 @@ +package ru.ifmo.torrent.messages; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public interface NetworkMessage { + + void write(DataOutputStream out) throws IOException; + + void read(DataInputStream in) throws IOException; +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/Request.java b/torrent/src/main/java/ru/ifmo/torrent/messages/Request.java new file mode 100644 index 0000000..8daa359 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/Request.java @@ -0,0 +1,18 @@ +package ru.ifmo.torrent.messages; + +import java.io.DataInputStream; +import java.io.IOException; + +public abstract class Request implements NetworkMessage { + + public abstract byte marker(); + + public abstract T getEmptyResponse(); + + public static Request readFromDataInputStream(DataInputStream in, Class cls) throws IllegalAccessException, InstantiationException, IOException { + Request request = cls.newInstance(); + request.read(in); + return request; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/Response.java b/torrent/src/main/java/ru/ifmo/torrent/messages/Response.java new file mode 100644 index 0000000..b399d41 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/Response.java @@ -0,0 +1,3 @@ +package ru.ifmo.torrent.messages; + +public abstract class Response implements NetworkMessage {} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/Marker.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/Marker.java new file mode 100644 index 0000000..a7682b8 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/Marker.java @@ -0,0 +1,8 @@ +package ru.ifmo.torrent.messages.client_tracker; + +public class Marker { + public static final byte LIST = 1; + public static final byte UPLOAD = 2; + public static final byte SOURCES = 3; + public static final byte UPDATE = 4; +} \ No newline at end of file diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/ListRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/ListRequest.java new file mode 100644 index 0000000..16e8905 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/ListRequest.java @@ -0,0 +1,33 @@ +package ru.ifmo.torrent.messages.client_tracker.requests; + +import ru.ifmo.torrent.messages.client_tracker.Marker; +import ru.ifmo.torrent.messages.client_tracker.response.ListResponse; +import ru.ifmo.torrent.messages.Request; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class ListRequest extends Request { + + public ListRequest() {} + + @Override + public byte marker() { + return Marker.LIST; + } + + @Override + public ListResponse getEmptyResponse() { + return new ListResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + } + + @Override + public void read(DataInputStream in) {} + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/SourcesRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/SourcesRequest.java new file mode 100644 index 0000000..e64b71a --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/SourcesRequest.java @@ -0,0 +1,47 @@ +package ru.ifmo.torrent.messages.client_tracker.requests; + +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.client_tracker.Marker; +import ru.ifmo.torrent.messages.client_tracker.response.SourcesResponse; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class SourcesRequest extends Request { + + private int fileId; + + public SourcesRequest() {} + + public SourcesRequest(int fileId) { + this.fileId = fileId; + } + + @Override + public byte marker() { + return Marker.SOURCES; + } + + @Override + public SourcesResponse getEmptyResponse() { + return new SourcesResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + out.writeInt(fileId); + } + + @Override + public void read(DataInputStream in) throws IOException { + fileId = in.readInt(); + + } + + public int getFileId() { + return fileId; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UpdateRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UpdateRequest.java new file mode 100644 index 0000000..d36d8f9 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UpdateRequest.java @@ -0,0 +1,64 @@ +package ru.ifmo.torrent.messages.client_tracker.requests; + +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.client_tracker.Marker; +import ru.ifmo.torrent.messages.client_tracker.response.UpdateResponse; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +public class UpdateRequest extends Request { + private short clientPort; + private List fileIds; + + public UpdateRequest() { + fileIds = new ArrayList<>(); + } + + public UpdateRequest(short clientPort, List fileIds) { + this.clientPort = clientPort; + this.fileIds = fileIds; + } + + @Override + public byte marker() { + return Marker.UPDATE; + } + + @Override + public UpdateResponse getEmptyResponse() { + return new UpdateResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + out.writeShort(clientPort); + out.writeInt(fileIds.size()); + for (Integer id : fileIds) { + out.writeInt(id); + } + } + + @Override + public void read(DataInputStream in) throws IOException { + clientPort = in.readShort(); + int count = in.readInt(); + for (int i = 0; i < count; i++) { + int fileId = in.readInt(); + fileIds.add(fileId); + } + } + + public short getClientPort() { + return clientPort; + } + + public List getFileIds() { + return fileIds; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UploadRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UploadRequest.java new file mode 100644 index 0000000..53b0a0d --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/requests/UploadRequest.java @@ -0,0 +1,57 @@ +package ru.ifmo.torrent.messages.client_tracker.requests; + +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.client_tracker.Marker; +import ru.ifmo.torrent.messages.client_tracker.response.UploadResponse; +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; + +public class UploadRequest extends Request { + + private String fileName; + private long fileSize; + + public UploadRequest() {} + + public UploadRequest(Path file) throws IOException { + this.fileName = file.getFileName().toString(); + this.fileSize = Files.size(file); + } + + @Override + public byte marker() { + return Marker.UPLOAD; + } + + @Override + public UploadResponse getEmptyResponse() { + return new UploadResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + out.writeUTF(fileName); + out.writeLong(fileSize); + } + + @Override + public void read(DataInputStream in) throws IOException { + fileName = in.readUTF(); + fileSize = in.readLong(); + } + + public String getFileName() { + return fileName; + } + + public long getFileSize() { + return fileSize; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/ListResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/ListResponse.java new file mode 100644 index 0000000..3f1ca37 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/ListResponse.java @@ -0,0 +1,49 @@ +package ru.ifmo.torrent.messages.client_tracker.response; + +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.tracker.state.FileInfo; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +public class ListResponse extends Response { + private List files; + + public ListResponse() { + files = new ArrayList<>(); + } + + public ListResponse(List files) { + this.files = files; + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeInt(files.size()); + for (FileInfo f : files) { + out.writeInt(f.getId()); + out.writeUTF(f.getName()); + out.writeLong(f.getSize()); + } + } + + @Override + public void read(DataInputStream in) throws IOException { + int count = in.readInt(); + for (int i = 0; i < count; i++) { + int id = in.readInt(); + String name = in.readUTF(); + long size = in.readLong(); + FileInfo f = new FileInfo(id, name, size); + files.add(f); + } + + } + + public List getFiles() { + return files; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/SourcesResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/SourcesResponse.java new file mode 100644 index 0000000..2f495ea --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/SourcesResponse.java @@ -0,0 +1,57 @@ +package ru.ifmo.torrent.messages.client_tracker.response; + +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.tracker.state.SeedInfo; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +public class SourcesResponse extends Response { + private int fileId; + private List clients; + + public SourcesResponse() { + clients = new ArrayList<>(); + } + + public SourcesResponse(int fileId, List clients) { + this.fileId = fileId; + this.clients = clients; + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeInt(clients.size()); + for (SeedInfo c : clients) { + out.write(c.getIP()); + out.writeShort(c.getPort()); + } + } + + @Override + public void read(DataInputStream in) throws IOException { + int count = in.readInt(); + for (int i = 0; i < count; i++) { + byte[] IP = new byte[4]; + in.readFully(IP); + short port = in.readShort(); + clients.add(new SeedInfo(port, IP)); + } + } + + public int getFileId() { + return fileId; + } + + public List getClients() { + return clients; + } + + public void setFileId(int fileId) { + this.fileId = fileId; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UpdateResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UpdateResponse.java new file mode 100644 index 0000000..708bbb9 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UpdateResponse.java @@ -0,0 +1,33 @@ +package ru.ifmo.torrent.messages.client_tracker.response; + +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class UpdateResponse extends Response { + private boolean success; + + public UpdateResponse() { + } + + public UpdateResponse(boolean success) { + this.success = success; + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeBoolean(success); + } + + @Override + public void read(DataInputStream in) throws IOException { + success = in.readBoolean(); + } + + public boolean getResult() { + return success; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UploadResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UploadResponse.java new file mode 100644 index 0000000..231f5f0 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/client_tracker/response/UploadResponse.java @@ -0,0 +1,34 @@ +package ru.ifmo.torrent.messages.client_tracker.response; + +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class UploadResponse extends Response { + + private int fileId; + + public UploadResponse() { + } + + public UploadResponse(int fileId) { + this.fileId = fileId; + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeInt(fileId); + } + + @Override + public void read(DataInputStream in) throws IOException { + fileId = in.readInt(); + } + + public int getFileId() { + return fileId; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/Marker.java b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/Marker.java new file mode 100644 index 0000000..870396d --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/Marker.java @@ -0,0 +1,6 @@ +package ru.ifmo.torrent.messages.seed_peer; + +public class Marker { + public static final byte STAT = 1; + public static final byte GET = 2; +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/GetRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/GetRequest.java new file mode 100644 index 0000000..4766d2a --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/GetRequest.java @@ -0,0 +1,56 @@ +package ru.ifmo.torrent.messages.seed_peer.requests; + +import ru.ifmo.torrent.messages.seed_peer.Marker; +import ru.ifmo.torrent.messages.seed_peer.response.GetResponse; +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class GetRequest extends Request { + + private int fileId; + private int part; + + public GetRequest() { + } + + public GetRequest(int fileId, int part) { + this.fileId = fileId; + this.part = part; + } + + @Override + public byte marker() { + return Marker.GET; + } + + @Override + public GetResponse getEmptyResponse() { + return new GetResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + out.writeInt(fileId); + out.writeInt(part); + } + + @Override + public void read(DataInputStream in) throws IOException { + fileId = in.readInt(); + part = in.readInt(); + } + + public int getFileId() { + return fileId; + } + + public int getPart() { + return part; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/StatRequest.java b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/StatRequest.java new file mode 100644 index 0000000..e174ce6 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/requests/StatRequest.java @@ -0,0 +1,47 @@ +package ru.ifmo.torrent.messages.seed_peer.requests; + +import ru.ifmo.torrent.messages.seed_peer.Marker; +import ru.ifmo.torrent.messages.seed_peer.response.StatResponse; +import ru.ifmo.torrent.messages.Request; +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; + +public class StatRequest extends Request { + private int fileId; + + public StatRequest() { + } + + public StatRequest(int fileId) { + this.fileId = fileId; + } + + @Override + public byte marker() { + return Marker.STAT; + } + + @Override + public StatResponse getEmptyResponse() { + return new StatResponse(); + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeByte(marker()); + out.writeInt(fileId); + } + + @Override + public void read(DataInputStream in) throws IOException { + fileId = in.readInt(); + } + + public int getFileId() { + return fileId; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/GetResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/GetResponse.java new file mode 100644 index 0000000..ddc7196 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/GetResponse.java @@ -0,0 +1,40 @@ +package ru.ifmo.torrent.messages.seed_peer.response; + +import org.apache.commons.io.IOUtils; +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.messages.Response; + +import java.io.*; + +public class GetResponse extends Response { + + private byte[] content; + private int size; + + public GetResponse() {} + + public GetResponse(InputStream in, int size) throws IOException { + this.size = size; + this.content = new byte[size]; + int totalReaded = 0; + while(totalReaded != size) { + totalReaded += in.read(content); + } + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.write(content); + } + + @Override + public void read(DataInputStream in) throws IOException { + ByteArrayOutputStream buffer = new ByteArrayOutputStream(size); + IOUtils.copy(in, buffer); + content = buffer.toByteArray(); + } + + public byte[] getContent() { + return content; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/StatResponse.java b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/StatResponse.java new file mode 100644 index 0000000..bb509ea --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/messages/seed_peer/response/StatResponse.java @@ -0,0 +1,41 @@ +package ru.ifmo.torrent.messages.seed_peer.response; + +import ru.ifmo.torrent.messages.Response; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +public class StatResponse extends Response { + + private List availableParts; + + public StatResponse() {} + + public StatResponse(List availableParts) { + this.availableParts = availableParts; + } + + @Override + public void write(DataOutputStream out) throws IOException { + out.writeInt(availableParts.size()); + for (Integer i: availableParts) { + out.writeInt(i); + } + } + + @Override + public void read(DataInputStream in) throws IOException { + availableParts = new ArrayList<>(); + int count = in.readInt(); + for (int i = 0; i < count; i++) { + availableParts.add(in.readInt()); + } + } + + public List getAvailableParts() { + return availableParts; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/ClientHandler.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/ClientHandler.java new file mode 100644 index 0000000..3bed396 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/ClientHandler.java @@ -0,0 +1,85 @@ +package ru.ifmo.torrent.tracker; + +import ru.ifmo.torrent.messages.client_tracker.Marker; +import ru.ifmo.torrent.messages.client_tracker.requests.*; +import ru.ifmo.torrent.messages.client_tracker.response.*; +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.tracker.state.FileInfo; +import ru.ifmo.torrent.tracker.state.SeedInfo; +import ru.ifmo.torrent.tracker.state.TrackerState; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.net.InetAddress; +import java.net.Socket; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; + +public class ClientHandler implements Runnable { + + private final Socket client; + private final TrackerState trackerState; + + ClientHandler(Socket client, TrackerState trackerState) { + this.client = client; + this.trackerState = trackerState; + } + + @Override + public void run() { + try (Socket clientSocket = client) { + DataOutputStream out = new DataOutputStream(clientSocket.getOutputStream()); + DataInputStream in = new DataInputStream(clientSocket.getInputStream()); + + Response response = null; + int marker; + while ((marker = in.read()) != -1) { + switch (marker) { + case Marker.LIST: + response = new ListResponse(trackerState.getAvailableFiles()); + break; + case Marker.SOURCES: { + SourcesRequest request = (SourcesRequest) SourcesRequest.readFromDataInputStream(in, SourcesRequest.class); + int fileId = request.getFileId(); + response = new SourcesResponse(fileId, trackerState.getSources(fileId)); + break; + } + case Marker.UPDATE: { + UpdateRequest request = (UpdateRequest) UpdateRequest.readFromDataInputStream(in, UpdateRequest.class); + InetAddress address = InetAddress.getByName(clientSocket.getInetAddress().getHostAddress()); + SeedInfo newSeed = new SeedInfo(request.getClientPort(), address); + boolean success = update(request.getFileIds(), newSeed); + response = new UpdateResponse(success); + break; + } + case Marker.UPLOAD: { + UploadRequest request = (UploadRequest) UploadRequest.readFromDataInputStream(in, UploadRequest.class); + int fileId = trackerState.addFile(request.getFileName(), request.getFileSize()); + response = new UploadResponse(fileId); + break; + } + default: + break; + } + if (response != null) { + response.write(out); + out.flush(); + } + } + } catch (Exception e) { + throw new IllegalStateException("error on tracker acquired", e); + } + } + + private boolean update(List fileIds, SeedInfo newSeed) { + Set allFiles = trackerState.getAvailableFiles().stream() + .map(FileInfo::getId) + .collect(Collectors.toSet()); + + if (!allFiles.containsAll(fileIds)) return false; + + fileIds.forEach(id -> trackerState.addNewSeedIfAbsent(id, newSeed)); + return true; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/Tracker.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/Tracker.java new file mode 100644 index 0000000..937a3ce --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/Tracker.java @@ -0,0 +1,57 @@ +package ru.ifmo.torrent.tracker; + +import ru.ifmo.torrent.tracker.state.TrackerState; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.net.ServerSocket; +import java.net.Socket; +import java.nio.file.Path; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; + +public class Tracker implements AutoCloseable, Runnable { + + private final ExecutorService pool = Executors.newFixedThreadPool(TrackerConfig.THREADS_COUNT); + private final TrackerState state; + private final ServerSocket serverSocket; + + public Tracker(short port, Path metaDir) throws TorrentException, IOException { + state = new TrackerState(metaDir.resolve(TrackerConfig.TRACKER_STATE_FILE)); + + try { + serverSocket = new ServerSocket(port); + } catch (IOException e) { + throw new TorrentException("cannot open server socket", e); + } + } + + @Override + public void run() { + pool.submit(() -> { + try { + while (!Thread.interrupted()) { + Socket client = serverSocket.accept(); + pool.submit(new ClientHandler(client, state)); + } + } catch (IOException e) { + if (!serverSocket.isClosed()) { + throw new IllegalStateException("cannot close tracker socket", e); + } + } + }); + } + + @Override + public void close() throws TorrentException { + try { + serverSocket.close(); + } catch (IOException e) { + throw new TorrentException("cannot close tracker properly", e); + } finally { + pool.shutdown(); + state.storeToFile(); + } + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerApp.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerApp.java new file mode 100644 index 0000000..77e1d78 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerApp.java @@ -0,0 +1,31 @@ +package ru.ifmo.torrent.tracker; + +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.util.Scanner; + +public class TrackerApp { + + public static void main(String[] args) { + Scanner scanner = new Scanner(System.in); + + try (Tracker tracker = new Tracker(TrackerConfig.TRACKER_PORT, TrackerConfig.getMetaDir())) { + tracker.run(); + System.out.printf("tracker started at getPort %d%n", TrackerConfig.TRACKER_PORT); + System.out.println("enter 'exit' to shutdown tracker"); + while (scanner.hasNext()) { + String command = scanner.next(); + if (command.equals("exit")) { + System.out.println("shutting down tracker"); + break; + } + } + } catch (TorrentException e) { + System.out.println(e.getMassage()); + if(e.getException() != null) e.getException().printStackTrace(); + } catch (IOException e) { + e.printStackTrace(); + } + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerConfig.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerConfig.java new file mode 100644 index 0000000..b1e4036 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/TrackerConfig.java @@ -0,0 +1,27 @@ +package ru.ifmo.torrent.tracker; + +import ru.ifmo.torrent.util.Config; + +import java.nio.file.Files; +import java.nio.file.Path; + +public class TrackerConfig extends Config { + + public static final int TIMEOUT = 5 * 1000; + public static final short TRACKER_PORT = 8081; + public static final int THREADS_COUNT = 8; + public static final int UPDATE_RATE_SEC = 180; + + public static final String TRACKER_STATE_FILE = "tracker_state_file"; + + private TrackerConfig() { + } + + public static Path getMetaDir() { + Path storage = TRACKER_STORAGE; + if (Files.notExists(storage)) { + storage.toFile().mkdirs(); + } + return storage; + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/state/FileInfo.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/FileInfo.java new file mode 100644 index 0000000..925f691 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/FileInfo.java @@ -0,0 +1,59 @@ +package ru.ifmo.torrent.tracker.state; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.Objects; + +public class FileInfo { + + private int id; + private String name; + private long size; + + public FileInfo(int id, String name, long size) { + this.id = id; + this.name = name; + this.size = size; + } + + public int getId() { + return id; + } + + public long getSize() { + return size; + } + + public String getName() { + return name; + } + + public static FileInfo readFrom(DataInputStream in) throws IOException { + int id = in.readInt(); + String name = in.readUTF(); + long size = in.readLong(); + return new FileInfo(id, name, size); + } + + public void write(DataOutputStream out) throws IOException { + out.writeInt(id); + out.writeUTF(name); + out.writeLong(size); + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + FileInfo fileInfo = (FileInfo) o; + return id == fileInfo.id && + size == fileInfo.size && + Objects.equals(name, fileInfo.name); + } + + @Override + public int hashCode() { + return Objects.hash(id, name, size); + } +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/state/SeedInfo.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/SeedInfo.java new file mode 100644 index 0000000..5ab6e6c --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/SeedInfo.java @@ -0,0 +1,48 @@ +package ru.ifmo.torrent.tracker.state; + +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.util.Objects; + +public class SeedInfo { + + private final short port; + private final InetAddress inetAddress; + + public SeedInfo(short port, byte[] IP) throws UnknownHostException { + this.port = port; + this.inetAddress = InetAddress.getByAddress(IP); + } + + public SeedInfo(short port, InetAddress inetAddress) { + this.port = port; + this.inetAddress = inetAddress; + } + + public byte[] getIP() { + return inetAddress.getAddress(); + } + + public short getPort() { + return port; + } + + public InetAddress getInetAddress() { + return inetAddress; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + SeedInfo seedInfo = (SeedInfo) o; + return port == seedInfo.port && + Objects.equals(inetAddress, seedInfo.inetAddress); + } + + @Override + public int hashCode() { + return Objects.hash(port, inetAddress); + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TimedSeedInfo.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TimedSeedInfo.java new file mode 100644 index 0000000..5ad7b79 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TimedSeedInfo.java @@ -0,0 +1,38 @@ +package ru.ifmo.torrent.tracker.state; + +import ru.ifmo.torrent.tracker.TrackerConfig; + +import java.util.Objects; + +public class TimedSeedInfo { + + private final SeedInfo seedInfo; + private final long creationTime; + + public TimedSeedInfo(SeedInfo seedInfo, long time) { + this.seedInfo = seedInfo; + this.creationTime = time; + } + + @Override + public boolean equals(Object o) { + if (this == o) return true; + if (o == null || getClass() != o.getClass()) return false; + TimedSeedInfo that = (TimedSeedInfo) o; + return Objects.equals(seedInfo, that.seedInfo); + } + + @Override + public int hashCode() { + return Objects.hash(seedInfo); + } + + public SeedInfo getSeedInfo() { + return seedInfo; + } + + public boolean notAlive(long currentTime) { + return currentTime - creationTime >= TrackerConfig.TIMEOUT; + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TrackerState.java b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TrackerState.java new file mode 100644 index 0000000..c6da621 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/tracker/state/TrackerState.java @@ -0,0 +1,109 @@ +package ru.ifmo.torrent.tracker.state; + +import ru.ifmo.torrent.tracker.TrackerConfig; +import ru.ifmo.torrent.util.StoredState; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Instant; +import java.util.*; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.Executors; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; + +public class TrackerState implements StoredState { + + private final ScheduledExecutorService pool; + private final ConcurrentHashMap availableFiles = new ConcurrentHashMap<>(); + private final ConcurrentHashMap> sources = new ConcurrentHashMap<>(); + + private final Path metaFile; + + public TrackerState(Path metaFile) throws TorrentException, IOException { + this.metaFile = metaFile; + if (Files.notExists(metaFile)) { + metaFile.getParent().toFile().mkdirs(); + Files.createFile(metaFile); + } + pool = Executors.newScheduledThreadPool(1); + pool.scheduleAtFixedRate(this::updateSeedList, 0, TrackerConfig.UPDATE_RATE_SEC, TimeUnit.SECONDS); + restoreFromFile(); + } + + public synchronized int addFile(String name, long size) { + int id = generateId(); + availableFiles.put(id, new FileInfo(id, name, size)); + sources.put(id, Collections.synchronizedSet(new HashSet<>())); + return id; + } + + public synchronized List getAvailableFiles() { + return new ArrayList<>(availableFiles.values()); + } + + private synchronized int generateId() { + return availableFiles.size() + 1; + } + + public synchronized void addNewSeedIfAbsent(int fileId, SeedInfo source) { + long currentTime = Instant.now().toEpochMilli(); + sources.computeIfAbsent(fileId, id -> + Collections.synchronizedSet(new HashSet<>())) + .add(new TimedSeedInfo(source, currentTime)); + } + + public synchronized List getSources(int fileId) { + return sources.getOrDefault(fileId, new HashSet<>()) + .stream() + .map(TimedSeedInfo::getSeedInfo) + .collect(Collectors.toList()); + } + + @Override + public synchronized void storeToFile() throws TorrentException { + pool.shutdown(); + try (DataOutputStream out = new DataOutputStream(Files.newOutputStream(metaFile))) { + out.writeInt(availableFiles.size()); + for (FileInfo info : availableFiles.values()) { + info.write(out); + } + out.flush(); + } catch (IOException e) { + throw new TorrentException("cannot save tracker state", e); + } + } + + @Override + public void restoreFromFile() throws TorrentException { + try { + if (Files.size(metaFile) == 0) return; + try (DataInputStream in = new DataInputStream(Files.newInputStream(metaFile))) { + int filesNumber = in.readInt(); + for (int i = 0; i < filesNumber; ++i) { + FileInfo fileInfo = FileInfo.readFrom(in); + availableFiles.put(fileInfo.getId(), fileInfo); + } + } + } catch (IOException e) { + throw new TorrentException("cannot restore torrent state", e); + } + } + + private void updateSeedList() { + long currentTime = Instant.now().toEpochMilli(); + for (Map.Entry> fileToSources : sources.entrySet()) { + + Set values = fileToSources.getValue(); + synchronized (values) { + values.removeIf(s -> s.notAlive(currentTime)); + } + } + } + +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/util/Config.java b/torrent/src/main/java/ru/ifmo/torrent/util/Config.java new file mode 100644 index 0000000..b27c9b8 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/util/Config.java @@ -0,0 +1,14 @@ +package ru.ifmo.torrent.util; + +import java.nio.file.Path; +import java.nio.file.Paths; + +public abstract class Config { + + public static Path CWD = Paths.get(System.getProperty("user.dir")).normalize(); + + public static final Path TORRENT_DIR = CWD.resolve("torrent"); + protected static final Path TORRENT_META_INFO_DIR = TORRENT_DIR.resolve(".metainfo"); + protected static final Path TRACKER_STORAGE = TORRENT_META_INFO_DIR.resolve("tracker"); + protected static final Path CLIENT_STORAGE = TORRENT_META_INFO_DIR.resolve("client"); +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/util/StoredState.java b/torrent/src/main/java/ru/ifmo/torrent/util/StoredState.java new file mode 100644 index 0000000..978bdf5 --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/util/StoredState.java @@ -0,0 +1,8 @@ +package ru.ifmo.torrent.util; + +public interface StoredState { + + void restoreFromFile() throws TorrentException; + + void storeToFile() throws TorrentException; +} diff --git a/torrent/src/main/java/ru/ifmo/torrent/util/TorrentException.java b/torrent/src/main/java/ru/ifmo/torrent/util/TorrentException.java new file mode 100644 index 0000000..b81e86b --- /dev/null +++ b/torrent/src/main/java/ru/ifmo/torrent/util/TorrentException.java @@ -0,0 +1,32 @@ +package ru.ifmo.torrent.util; + +import java.io.PrintStream; + +public class TorrentException extends Exception { + private final String message; + private final Throwable exception; + + public TorrentException(String message) { + super(message); + this.message = message; + exception = null; + } + + public TorrentException(String message, Throwable e) { + super(message); + this.message = message; + exception = e; + } + + public String getMassage() { + return message; + } + + public Throwable getException() { + return exception; + } + + public void write(PrintStream printer) { + printer.println(message); + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/TorrentTest.java b/torrent/src/test/java/ru/ifmo/torrent/TorrentTest.java new file mode 100644 index 0000000..64bac4c --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/TorrentTest.java @@ -0,0 +1,118 @@ +package ru.ifmo.torrent; + +import org.apache.commons.io.FileUtils; +import org.junit.*; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.client.Client; +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.tracker.Tracker; +import ru.ifmo.torrent.tracker.TrackerConfig; +import ru.ifmo.torrent.tracker.state.FileInfo; +import ru.ifmo.torrent.tracker.state.SeedInfo; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.net.InetAddress; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import static junit.framework.TestCase.assertTrue; +import static org.assertj.core.api.Assertions.assertThat; + +public class TorrentTest { + private final int port1 = 1111; + private final int port2 = 1199; + private Tracker tracker; + private Client client1; + private Client client2; + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + private Path folder1; + private Path folder2; + private Path downloadFolder1; + private Path downloadFolder2; + + @Before + public void runTorrent() throws IOException, TorrentException { + folder1 = folder.newFolder("client1").toPath(); + folder2 = folder.newFolder("client2").toPath(); + downloadFolder1 = folder.newFolder("downloads1").toPath(); + downloadFolder2 = folder.newFolder("downloads2").toPath(); + + tracker = new Tracker(TrackerConfig.TRACKER_PORT, folder1); + tracker.run(); + + client1 = new Client(InetAddress.getLocalHost(), (short) port1, folder1, downloadFolder1); + client2 = new Client(InetAddress.getLocalHost(), (short) port2, folder2, downloadFolder2); + } + + @After + public void stopTorrent() throws TorrentException { + tracker.close(); + client1.close(); + client2.close(); + } + + @Test + public void testEmptyListRequest() throws IOException, TorrentException { + List availableFiles = client1.getAvailableFiles(); + assertThat(availableFiles).isEmpty(); + } + + @Test + public void testUploadAndListRequest() throws IOException, TorrentException { + Path file = createFile(); + addFileToTrackerAndCheckContains(file); + } + + @Test + public void testUploadFileAndSources() throws IOException, TorrentException, InterruptedException { + Path file = createFile(); + FileInfo addedFile = addFileToTrackerAndCheckContains(file); + + Thread.sleep((ClientConfig.UPDATE_RATE_SEC + 1) * 1000); + List sources = client1.getFileSources(addedFile.getId()); + + SeedInfo seedAdded = new SeedInfo((short) port1, InetAddress.getByName("localhost")); + assertThat(sources.size()).isEqualTo(1); + assertThat(sources.get(0)).isEqualTo(seedAdded); + } + + @Test + public void testDownloadFile() throws IOException, TorrentException, InterruptedException { + Path file = createFile(); + FileInfo addedFile = addFileToTrackerAndCheckContains(file); + + Thread.sleep((ClientConfig.UPDATE_RATE_SEC + 1) * 1000); + boolean result = client2.downloadFile(addedFile.getId()); + Thread.sleep((ClientConfig.DOWNLOAD_RATE_SEC + 1) * 1000); + + Path downloadedFile = downloadFolder2.resolve(file.getFileName()); + assertTrue(result); + assertTrue(Files.exists(downloadedFile)); + assertThat(Files.size(downloadedFile)).isEqualTo(Files.size(file)); + assertThat(FileUtils.readFileToString(downloadedFile.toFile())) + .isEqualTo(FileUtils.readFileToString(file.toFile())); + } + + private FileInfo addFileToTrackerAndCheckContains(Path file) throws IOException, TorrentException { + int id = client1.uploadFile(file); + List availableFiles = client1.getAvailableFiles(); + assertThat(availableFiles.size()).isEqualTo(1); + + FileInfo addedFile = new FileInfo(id, file.getFileName().toString(), Files.size(file)); + assertThat(availableFiles.get(0)).isEqualTo(addedFile); + + return addedFile; + } + + private Path createFile() throws IOException { + Path file = folder1.resolve("fileName"); + Files.createFile(file); + FileUtils.writeStringToFile(file.toFile(), "kek!"); + return file; + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/client/storage/LocalFilesManagerTest.java b/torrent/src/test/java/ru/ifmo/torrent/client/storage/LocalFilesManagerTest.java new file mode 100644 index 0000000..e8bf69a --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/client/storage/LocalFilesManagerTest.java @@ -0,0 +1,61 @@ +package ru.ifmo.torrent.client.storage; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.client.ClientConfig; +import ru.ifmo.torrent.client.storage.LocalFilesManager; +import ru.ifmo.torrent.client.storage.LocalFileReference; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.nio.file.Path; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class LocalFilesManagerTest { + + private static final long size = 17; + private static final int id = 0; + private static final String name = "kek"; + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + @Test + public void testEmptyState() throws IOException, TorrentException { + LocalFilesManager state = new LocalFilesManager(folder.newFolder().toPath(), folder.newFile().toPath(), folder.newFolder().toPath()); + assertThat(state.getFiles()).isEmpty(); + } + + @Test + public void testAddAndGet() throws IOException, TorrentException { + LocalFilesManager state = new LocalFilesManager(folder.newFolder().toPath(), folder.newFile().toPath(), folder.newFolder().toPath()); + state.addLocalFile(name, id, size); + testGetFile(state); + } + + @Test + public void testAddAndContainsAfterReloading() throws IOException, TorrentException { + Path metaFile = folder.newFile(ClientConfig.LOCAL_FILES_FILE).toPath(); + Path partsDir = folder.newFolder(ClientConfig.PARTS_STORAGE).toPath(); + LocalFilesManager storedState = new LocalFilesManager(folder.newFolder().toPath(), metaFile, partsDir); + + storedState.addLocalFile(name, id, size); + testGetFile(storedState); + storedState.storeToFile(); + + LocalFilesManager restoredState = new LocalFilesManager(folder.newFolder().toPath(), metaFile, partsDir); + restoredState.restoreFromFile(); + testGetFile(restoredState); + } + + private void testGetFile(LocalFilesManager state) { + LocalFileReference fileState = state.getFileReference(id); + assertThat(fileState.getFileId()).isEqualTo(id); + assertThat(fileState.getName()).isEqualTo(name); + assertThat(fileState.getNumberOfParts()).isEqualTo(1); + } + +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/client/storage/PartsManagerTest.java b/torrent/src/test/java/ru/ifmo/torrent/client/storage/PartsManagerTest.java new file mode 100644 index 0000000..9bfb193 --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/client/storage/PartsManagerTest.java @@ -0,0 +1,59 @@ +package ru.ifmo.torrent.client.storage; + +import org.apache.commons.io.FileUtils; +import org.apache.commons.io.IOUtils; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.client.storage.LocalFileReference; +import ru.ifmo.torrent.client.storage.PartsManager; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +public class PartsManagerTest { + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + @Test + public void testStoreSplittedAndThenReading() throws IOException { + Path file = folder.newFile().toPath(); + String content = "content"; + FileUtils.writeStringToFile(file.toFile(), content); + PartsManager partsManager = new PartsManager(folder.getRoot().toPath()); + + int id = 0; + LocalFileReference reference = LocalFileReference.createEmpty(file.getFileName().toString(), 0, Files.size(file), 1); + partsManager.storeSplitted(reference, file); + String storedContent = IOUtils.toString(partsManager.getForReading(id,0)); + assertThat(storedContent).isEqualTo(content); + } + + @Test + public void testMergeParts() throws IOException { + Path fileDir = folder.newFolder("0").toPath(); + PartsManager partsManager = new PartsManager(folder.getRoot().toPath()); + List files = Arrays.asList( + Files.createFile(fileDir.resolve("0")), + Files.createFile(fileDir.resolve("1")), + Files.createFile(fileDir.resolve("2")) + ); + List contents = Arrays.asList("content1", "content2", "content3"); + + for (int i = 0; i < contents.size(); i++) { + FileUtils.writeStringToFile(files.get(i).toFile(), contents.get(i)); + } + + Path mergedFile = folder.newFile().toPath(); + partsManager.mergeSplitted(0, 24, mergedFile); + + String storedContent = FileUtils.readFileToString(mergedFile.toFile()); + assertThat(storedContent).isEqualTo("content1content2content3"); + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/RequestsTests.java b/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/RequestsTests.java new file mode 100644 index 0000000..dda60ea --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/RequestsTests.java @@ -0,0 +1,82 @@ +package ru.ifmo.torrent.messages.client_tracker; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.messages.client_tracker.requests.*; +import ru.ifmo.torrent.messages.Request; + +import java.io.*; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; + +public class RequestsTests { + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + private ByteArrayOutputStream baos = new ByteArrayOutputStream(); + private DataOutputStream out = new DataOutputStream(baos); + + @Test + public void testListRequest() throws IOException, InstantiationException, IllegalAccessException { + ListRequest sentRequest = new ListRequest(); + DataInputStream in = testSendAndAccept(sentRequest, Marker.LIST); + ListRequest acceptedRequest = new ListRequest(); + + acceptedRequest.read(in); + int EOF = in.read(); + assertThat(EOF).isEqualTo(-1); + } + + @Test + public void testSourceRequest() throws IOException, InstantiationException, IllegalAccessException { + SourcesRequest sentRequest = new SourcesRequest(0); + SourcesRequest acceptedRequest = new SourcesRequest(); + DataInputStream in = testSendAndAccept(sentRequest, Marker.SOURCES); + + acceptedRequest.read(in); + assertThat(sentRequest.getFileId()).isEqualTo(acceptedRequest.getFileId()); + } + + @Test + public void testUpdateRequest() throws IOException, InstantiationException, IllegalAccessException { + List fileIDs = Arrays.asList(0, 1, 2); + UpdateRequest sentRequest = new UpdateRequest((short) 1111, fileIDs); + UpdateRequest acceptedRequest = new UpdateRequest(); + + DataInputStream in = testSendAndAccept(sentRequest, Marker.UPDATE); + acceptedRequest.read(in); + assertThat(acceptedRequest.getClientPort()).isEqualTo(sentRequest.getClientPort()); + assertThat(acceptedRequest.getFileIds().size()).isEqualTo(sentRequest.getFileIds().size()); + for (int i = 0; i < acceptedRequest.getFileIds().size(); i++) { + assertThat(acceptedRequest.getFileIds().get(i)).isEqualTo(sentRequest.getFileIds().get(i)); + } + } + + @Test + public void testUploadRequest() throws IOException, InstantiationException, IllegalAccessException { + Path tmpFile = folder.newFile("file_name").toPath(); + + UploadRequest sentRequest = new UploadRequest(tmpFile); + UploadRequest acceptedRequest = new UploadRequest(); + + DataInputStream in = testSendAndAccept(sentRequest, Marker.UPLOAD); + acceptedRequest.read(in); + assertThat(acceptedRequest.getFileName()).isEqualTo(sentRequest.getFileName()); + assertThat(acceptedRequest.getFileSize()).isEqualTo(sentRequest.getFileSize()); + } + + private DataInputStream testSendAndAccept(Request request, byte marker) throws IOException, IllegalAccessException, InstantiationException { + request.write(out); + out.flush(); + + DataInputStream in = new DataInputStream(new ByteArrayInputStream(baos.toByteArray())); + byte acceptedMarker = in.readByte(); + assertThat(acceptedMarker).isEqualTo(marker); + return in; + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/ResponseTests.java b/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/ResponseTests.java new file mode 100644 index 0000000..eab043c --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/messages/client_tracker/ResponseTests.java @@ -0,0 +1,89 @@ +package ru.ifmo.torrent.messages.client_tracker; + +import org.junit.Test; +import ru.ifmo.torrent.messages.client_tracker.response.*; +import ru.ifmo.torrent.messages.Response; +import ru.ifmo.torrent.tracker.state.FileInfo; +import ru.ifmo.torrent.tracker.state.SeedInfo; + +import java.io.*; +import java.net.InetAddress; +import java.util.Arrays; +import java.util.List; + +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; + +public class ResponseTests { + + private ByteArrayOutputStream baos = new ByteArrayOutputStream(); + private DataOutputStream out = new DataOutputStream(baos); + + @Test + public void testListResponse() throws IOException { + List files = Arrays.asList( + new FileInfo(0, "file_1", 1), + new FileInfo(1, "file_2", 100), + new FileInfo(2, "file_3", 17) + ); + + ListResponse sentResponse = new ListResponse(files); + ListResponse acceptedResponse = new ListResponse(); + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getFiles().size()).isEqualTo(files.size()); + for (int i = 0; i < files.size(); i++) { + FileInfo f = acceptedResponse.getFiles().get(i); + assertThat(f.getId()).isEqualTo(files.get(i).getId()); + assertThat(f.getName()).isEqualTo(files.get(i).getName()); + assertThat(f.getSize()).isEqualTo(files.get(i).getSize()); + } + } + + @Test + public void testSourceResponse() throws IOException { + List seeds = Arrays.asList( + new SeedInfo((short) 1111, InetAddress.getLocalHost()), + new SeedInfo((short) 1212, InetAddress.getLocalHost()), + new SeedInfo((short) 1313, InetAddress.getLocalHost()) + ); + SourcesResponse sentResponse = new SourcesResponse(0, seeds); + SourcesResponse acceptedResponse = new SourcesResponse(); + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getFileId()).isEqualTo(sentResponse.getFileId()); + assertThat(acceptedResponse.getClients().size()).isEqualTo(seeds.size()); + for (int i = 0; i < seeds.size(); i++) { + SeedInfo s = acceptedResponse.getClients().get(i); + assertThat(s.getInetAddress()).isEqualTo(seeds.get(i).getInetAddress()); + assertThat(s.getPort()).isEqualTo(seeds.get(i).getPort()); + } + } + + @Test + public void testUpdateResponse() throws IOException { + UpdateResponse sentResponse = new UpdateResponse(true); + UpdateResponse acceptedResponse = new UpdateResponse(); + + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getResult()).isEqualTo(sentResponse.getResult()); + } + + @Test + public void testUploadResponse() throws IOException { + UploadResponse sentResponse = new UploadResponse(17); + UploadResponse acceptedResponse = new UploadResponse(); + + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getFileId()).isEqualTo(sentResponse.getFileId()); + } + + private void sendAndAccept(Response sentResponse, Response acceptedResponse) throws IOException { + sentResponse.write(out); + out.flush(); + DataInputStream in = new DataInputStream(new ByteArrayInputStream(baos.toByteArray())); + acceptedResponse.read(in); + } + +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/RequestTests.java b/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/RequestTests.java new file mode 100644 index 0000000..385ef0b --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/RequestTests.java @@ -0,0 +1,51 @@ +package ru.ifmo.torrent.messages.seed_peer; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.messages.seed_peer.requests.GetRequest; +import ru.ifmo.torrent.messages.seed_peer.requests.StatRequest; +import ru.ifmo.torrent.messages.Request; + +import java.io.*; + +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; + +public class RequestTests { + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + private ByteArrayOutputStream baos = new ByteArrayOutputStream(); + private DataOutputStream out = new DataOutputStream(baos); + + @Test + public void testGetRequest() throws IOException { + GetRequest sentRequest = new GetRequest(17, 2); + DataInputStream in = testSendAndAccept(sentRequest, Marker.GET); + GetRequest acceptedRequest = new GetRequest(); + + acceptedRequest.read(in); + assertThat(acceptedRequest.getFileId()).isEqualTo(sentRequest.getFileId()); + assertThat(acceptedRequest.getPart()).isEqualTo(sentRequest.getPart()); + } + + @Test + public void testStatRequest() throws IOException { + StatRequest sentRequest = new StatRequest(17); + StatRequest acceptedRequest = new StatRequest(); + DataInputStream in = testSendAndAccept(sentRequest, Marker.STAT); + + acceptedRequest.read(in); + assertThat(sentRequest.getFileId()).isEqualTo(acceptedRequest.getFileId()); + } + + private DataInputStream testSendAndAccept(Request request, byte marker) throws IOException { + request.write(out); + out.flush(); + + DataInputStream in = new DataInputStream(new ByteArrayInputStream(baos.toByteArray())); + byte acceptedMarker = in.readByte(); + assertThat(acceptedMarker).isEqualTo(marker); + return in; + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/ResponseTests.java b/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/ResponseTests.java new file mode 100644 index 0000000..a809734 --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/messages/seed_peer/ResponseTests.java @@ -0,0 +1,53 @@ +package ru.ifmo.torrent.messages.seed_peer; + +import org.apache.commons.io.FileUtils; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.messages.seed_peer.response.GetResponse; +import ru.ifmo.torrent.messages.seed_peer.response.StatResponse; +import ru.ifmo.torrent.messages.Response; + +import java.io.*; +import java.nio.file.Files; +import java.util.Arrays; + +import static org.assertj.core.api.Assertions.assertThat; + + +public class ResponseTests { + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + private ByteArrayOutputStream baos = new ByteArrayOutputStream(); + private DataOutputStream out = new DataOutputStream(baos); + + @Test + public void testGetResponse() throws IOException { + File file = folder.newFile(); + FileUtils.writeStringToFile(file, "contentcontentcontentcontent"); + + GetResponse sentResponse = new GetResponse(Files.newInputStream(file.toPath()), 21); + GetResponse acceptedResponse = new GetResponse(); + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getContent()).isEqualTo(sentResponse.getContent()); + } + + @Test + public void testStatResponse() throws IOException { + StatResponse sentResponse = new StatResponse(Arrays.asList(0, 1, 2, 3, 4)); + StatResponse acceptedResponse = new StatResponse(); + sendAndAccept(sentResponse, acceptedResponse); + + assertThat(acceptedResponse.getAvailableParts()).containsExactlyElementsOf(sentResponse.getAvailableParts()); + } + + private void sendAndAccept(Response sentResponse, Response acceptedResponse) throws IOException { + sentResponse.write(out); + out.flush(); + DataInputStream in = new DataInputStream(new ByteArrayInputStream(baos.toByteArray())); + acceptedResponse.read(in); + } +} diff --git a/torrent/src/test/java/ru/ifmo/torrent/tracker/state/TrackerStateTest.java b/torrent/src/test/java/ru/ifmo/torrent/tracker/state/TrackerStateTest.java new file mode 100644 index 0000000..5169ccf --- /dev/null +++ b/torrent/src/test/java/ru/ifmo/torrent/tracker/state/TrackerStateTest.java @@ -0,0 +1,67 @@ +package ru.ifmo.torrent.tracker.state; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import ru.ifmo.torrent.tracker.TrackerConfig; +import ru.ifmo.torrent.util.TorrentException; + +import java.io.IOException; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.List; + +import static junit.framework.TestCase.assertTrue; +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; + +public class TrackerStateTest { + + private final List files = Arrays.asList( + new FileInfo(1, "file_1", 1), + new FileInfo(2, "file_2", 100), + new FileInfo(3, "file_3", 17) + ); + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + private Path createMetaFile() throws IOException { + return folder.newFile(TrackerConfig.TRACKER_STATE_FILE).toPath(); + } + + @Test + public void testStoringAndRestoringState() throws IOException, TorrentException { + Path file = createMetaFile(); + TrackerState storedState = new TrackerState(file); + files.forEach(f -> storedState.addFile(f.getName(), f.getSize())); + storedState.storeToFile(); + + TrackerState restoredState = new TrackerState(file); + List restoredFiles = restoredState.getAvailableFiles(); + + assertThat(files.size()).isEqualTo(restoredFiles.size()); + for (int i = 0; i < files.size(); i++) { + assertThat(restoredFiles.get(i).getId()).isEqualTo(files.get(i).getId()); + assertThat(restoredFiles.get(i).getName()).isEqualTo(files.get(i).getName()); + assertThat(restoredFiles.get(i).getSize()).isEqualTo(files.get(i).getSize()); + } + } + + @Test + public void addAndContainsFileTest() throws IOException, TorrentException { + Path file = createMetaFile(); + TrackerState state = new TrackerState(file); + assertTrue(state.getAvailableFiles().isEmpty()); + + String fileName = "kek"; + long fileSize = 17; + int ID = state.addFile(fileName, fileSize); + assertThat(state.getAvailableFiles().size()).isEqualTo(1); + + FileInfo addedFile = state.getAvailableFiles().get(0); + assertThat(addedFile.getId()).isEqualTo(ID); + assertThat(addedFile.getName()).isEqualTo(fileName); + assertThat(addedFile.getSize()).isEqualTo(fileSize); + } + +} From 55d1a7b7e820a79fba4a30d92a5c35c764a70dfd Mon Sep 17 00:00:00 2001 From: lergor Date: Mon, 18 Mar 2019 01:21:46 +0300 Subject: [PATCH 4/4] change readme --- torrent/Readme.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/torrent/Readme.md b/torrent/Readme.md index fb8b9ef..6800c73 100644 --- a/torrent/Readme.md +++ b/torrent/Readme.md @@ -107,10 +107,10 @@ clientPort — порт клиента * склонировать [репозиторий](https://github.com/lergor/NetworksLab2019HSE.git) * переключиться на нужную ветку: `git co torrent` +* зайти в директорию `torrent` * запустить команды `gradle trackerJar` и `gradle clientJar` для сборки приложений трекера и клиента -- TrackerApp и ClientApp соответственно -* перейти в директорию `build/libs` -* запустить трекер: ```java -jar trackerApp-jar-1.0-SNAPSHOT.jar``` -* запустить клиент: ```java -jar clientApp-jar-1.0-SNAPSHOT.jar``` +* запустить трекер: ```java -jar build/libs/trackerApp-jar-1.0-SNAPSHOT.jar``` +* запустить клиент: ```java -jar build/libs/clientApp-jar-1.0-SNAPSHOT.jar``` ## Использование