Skip to content
This repository was archived by the owner on Oct 23, 2023. It is now read-only.

Fix wrapObject typeFooDerivedFromBar then unwrapObject typeBar will cause error on iOS #229

Open
flyskywhy wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Fix wrapObject typeFooDerivedFromBar then unwrapObject typeBar will cause error on iOS #229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
dd841fb
Fix wrapObject typeFooDerivedFromBar then unwrapObject typeBar will c…
flyskywhy Aug 31, 2023
ef8d59d
Add CameraModule.java so that await takePicture() 400ms is less than …
flyskywhy Sep 12, 2023
cb565ac
Support preview instead of photo on Android
flyskywhy Sep 14, 2023
5969a5b
Pass `bob build`
flyskywhy Dec 15, 2023
ad9cc4e
Change package name to @flyskywhy/react-native-pytorch-core
flyskywhy Dec 15, 2023
8740add
1.0.0
flyskywhy Dec 15, 2023
537dc20
Let @flyskywhy/react-native-pytorch-core be public
flyskywhy Dec 15, 2023
a5e544b
1.0.1
flyskywhy Dec 15, 2023
d2ecb83
Change package name from @flyskywhy/react-native-pytorch-core to reac…
flyskywhy Dec 18, 2023
191ae28
v1.0.0
flyskywhy Dec 18, 2023
71a8033
typo
flyskywhy Dec 18, 2023
e38ec45
Works well on iOS
flyskywhy Dec 20, 2023
fed32de
v1.0.1
flyskywhy Dec 20, 2023
52d76ee
typo
flyskywhy Dec 20, 2023
7c5fe6e
Support toImageDataDataWeb which export imageData.data with Web RGBA …
flyskywhy Mar 26, 2024
1f73e05
Fix RN0.71 cause `Expected directory '/home/foo/bar/android/../node_m…
flyskywhy Aug 7, 2024
8ae63d2
v1.0.2
flyskywhy Aug 7, 2024
e131854
Fix compile error `Duplicate class com.facebook.jni.CppException foun…
flyskywhy Aug 12, 2024
a05eed1
Fix Android NDK compile `ld: error: found local symbol '__end__'`
flyskywhy Aug 12, 2024
ce55e97
Add support for react-native 0.71 to fix `CMake Error: The following …
flyskywhy Aug 12, 2024
839e28c
Just change fbjni from 0.2.2 to 0.3.0 in expo-plugin, maybe not neces…
flyskywhy Aug 12, 2024
45d82ad
v1.0.3
flyskywhy Aug 12, 2024
08216a8
Fix `ninja: error: '../../../../build/pytorch_android_lite-1.12.2.aar…
flyskywhy Aug 12, 2024
7c3e581
v1.0.4
flyskywhy Aug 12, 2024
a8670be
README.md: Install
flyskywhy Aug 12, 2024
2bd648e
v1.0.5
flyskywhy Aug 12, 2024
8bcbb0c
Fix run crash on Android [java.lang.UnsatisfiedLinkError: dlopen fail…
flyskywhy Aug 15, 2024
4f63b11
v1.0.6
flyskywhy Aug 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 4 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,10 @@
<a href="https://playtorch.dev/docs/next/api/core/">API</a>
</h3>

<p>&nbsp;</p>

<p>
<a href="https://www.npmjs.org/package/react-native-pytorch-core">
<img src="https://img.shields.io/npm/v/react-native-pytorch-core?label=react-native-pytorch-core" alt="Current React Native PyTorch Core npm package version." />
</a>
</p>

<p>
<a href="https://github.com/pytorch/live/blob/main/LICENSE">
<img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="PlayTorch is released under the MIT license." />
</a>
</p>
[![npm version](http://img.shields.io/npm/v/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![npm downloads](http://img.shields.io/npm/dm/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![npm licence](http://img.shields.io/npm/l/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![Platform](https://img.shields.io/badge/platform-ios%20%7C%20android-989898.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")

----------------------

Expand Down
78 changes: 77 additions & 1 deletion react-native-pytorch-core/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# PyTorch core library for React Native

[![npm version](http://img.shields.io/npm/v/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![npm downloads](http://img.shields.io/npm/dm/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![npm licence](http://img.shields.io/npm/l/react-native-playtorch.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")
[![Platform](https://img.shields.io/badge/platform-ios%20%7C%20android-989898.svg?style=flat-square)](https://npmjs.org/package/react-native-playtorch "View this project on npm")

The PyTorch core library for React Native is part of the PlayTorch project. Please follow the instructions provided on the [PlayTorch website](https://playtorch.dev/) as outlined below!

## 🎉 Getting started with PlayTorch
Expand All @@ -13,6 +18,77 @@ Follow the [Getting Started guide](https://playtorch.dev/docs/tutorials/get-star

The full documentation for PlayTorch can be found on our [website](https://playtorch.dev/).

## Install
```
npm install react-native-playtorch
```

Modify `android/app/build.gradle`:
```
android {
....
packagingOptions {
// doNotStrip "**/libc++_shared.so"
pickFirst '**/*.so'
}
...
}
```
Modify `android/gradle.properties`:
```
org.gradle.jvmargs=-Xmx4g
```
Modify `metro.config.js`:
```
const defaultAssetExts = require('metro-config/src/defaults/defaults')
.assetExts;

module.exports = {
resolver: {
assetExts: [...defaultAssetExts, 'ptl'],
},
};
```

## Patch to fix `__emutls_get_address` crash on Android
If `RN0.71+` and run crash on Android `java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__emutls_get_address" referenced by "/data/app/~~Bu6UWdRieDpDrpvvyvNNVQ==/com.foo.bar-w8nusksLnLfSCCsWG3cEkg==/lib/arm64/libfolly_runtime.so"`, you need (e.g. on Linux)
```
cd tools/android-sdk/ndk

mv ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/origin_libc++_shared.so
cp ./23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/

mv ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/origin_libc++_shared.so
cp ./23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi/

mv ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/origin_libc++_shared.so
cp ./23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/x86_64-linux-android/

mv ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/i686-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/i686-linux-android/origin_libc++_shared.so
cp ./23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/i686-linux-android/libc++_shared.so ./21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/i686-linux-android/
```
The crash reason is, even the `libc++_shared.so` in `prefab`
```
readelf -s ~/.gradle/caches/transforms-3/9915f55612e7d9d2d8676faa1872c696/transformed/jetified-react-android-0.71.6-debug/jni/arm64-v8a/libc++_shared.so | grep __emutls_get_address
374: 00000000000ec5bc 448 FUNC WEAK DEFAULT 16 __emutls_get_address
```
is `WEAK` not `LOCAL`, but with `pickFirst '**/*.so'`, the `libc++_shared.so` in `.apk` will be picked from `node_modules/react-native-playtorch/android/build/intermediates/library_jni/debug/jni/arm64-v8a/libc++_shared.so`, and
```
readelf -s node_modules/react-native-playtorch/android/build/intermediates/library_jni/debug/jni/arm64-v8a/libc++_shared.so | grep __emutls_get_address
3885: 00000000000b60a0 344 FUNC LOCAL DEFAULT 11 __emutls_get_address
```
is `LOCAL` not `WEAK`.

If enable `doNotStrip "**/libc++_shared.so"` then extract the `libc++_shared.so` from `.apk`, use `readelf` you will also find it's `LOCAL` not `WEAK`.

The `libfolly_runtime.so` will call `__emutls_get_address`, if it's `LOCAL`, then run into crash.

The `__emutls_get_address` in `libc++_shared.so` of `NDK21.4.7075529` is `LOCAL`, and it's `WEAK` for `NDK23.1.7779620`.

For now, `react-native-playtorch` only can be compiled in `NDK21.4.7075529`.

So comes the patch above.

## Example Usage

```javascript
Expand All @@ -28,7 +104,7 @@ import {
Tensor,
torch,
torchvision,
} from 'react-native-pytorch-core';
} from 'react-native-playtorch';
import { useSafeAreaInsets } from 'react-native-safe-area-context';

// Alias for torchvision transforms
Expand Down
114 changes: 80 additions & 34 deletions react-native-pytorch-core/android/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,31 @@
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

project(PlayTorch)
cmake_minimum_required(VERSION 3.4.1)

set (CMAKE_VERBOSE_MAKEFILE ON)

# This should be matched with the Unit Test Build. Please keep them in sync. See ../cxx/test/CMakeLists.txt
set (CMAKE_CXX_STANDARD 14)
set (CMAKE_CXX_FLAGS "-DFOLLY_NO_CONFIG=1 -DFOLLY_HAVE_CLOCK_GETTIME=1 -DFOLLY_HAVE_MEMRCHR=1 -DFOLLY_USE_LIBCPP=1 -DFOLLY_MOBILE=1 -DON_ANDROID -DONANDROID")

if(${REACT_NATIVE_MINOR_VERSION} GREATER_EQUAL 71)
include("${NODE_MODULES_DIR}/react-native/ReactAndroid/cmake-utils/folly-flags.cmake")
add_compile_options(${folly_FLAGS})
else()
set (CMAKE_CXX_FLAGS "-DFOLLY_NO_CONFIG=1 -DFOLLY_HAVE_CLOCK_GETTIME=1 -DFOLLY_HAVE_MEMRCHR=1 -DFOLLY_USE_LIBCPP=1 -DFOLLY_MOBILE=1 -DON_ANDROID -DONANDROID")
endif()

set (PACKAGE_NAME "torchlive")
set (BUILD_DIR ${CMAKE_SOURCE_DIR}/build)
set (RN_SO_DIR ${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/jni/first-party/react/jni)

if(${REACT_NATIVE_MINOR_VERSION} GREATER_EQUAL 71)
# Consume shared libraries and headers from prefabs
find_package(fbjni REQUIRED CONFIG)
find_package(ReactAndroid REQUIRED CONFIG)
else()
set (RN_SO_DIR ${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/jni/first-party/react/jni)
endif()

# PyTorch Core shared

Expand Down Expand Up @@ -60,25 +74,43 @@ add_library(

# includes

file (GLOB LIBFBJNI_INCLUDE_DIR "${BUILD_DIR}/fbjni-*-headers.jar/")
file (GLOB PYTORCH_INCLUDE_DIRS "${BUILD_DIR}/pytorch_android_lite*.aar/headers")

target_include_directories(
${PACKAGE_NAME}
PRIVATE
"${LIBFBJNI_INCLUDE_DIR}"
"${NODE_MODULES_DIR}/react-native/React"
"${NODE_MODULES_DIR}/react-native/React/Base"
"${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/jni"
"${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/java/com/facebook/react/turbomodule/core/jni"
"${NODE_MODULES_DIR}/react-native/ReactCommon"
"${NODE_MODULES_DIR}/react-native/ReactCommon/callinvoker"
"${NODE_MODULES_DIR}/react-native/ReactCommon/jsi"
"${NODE_MODULES_DIR}/react-native/ReactCommon/runtimeexecutor"
"${PYTORCH_INCLUDE_DIRS}"
"../cxx/src"
"src/main/cpp"
)
if(${REACT_NATIVE_MINOR_VERSION} GREATER_EQUAL 71)
target_include_directories(
${PACKAGE_NAME}
PRIVATE
"${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/jni/react/turbomodule"
"${NODE_MODULES_DIR}/react-native/ReactCommon"
"${NODE_MODULES_DIR}/react-native/ReactCommon/callinvoker"
"${NODE_MODULES_DIR}/react-native/ReactCommon/jsi"
"${NODE_MODULES_DIR}/react-native/ReactCommon/react/renderer/graphics/platform/cxx"
"${NODE_MODULES_DIR}/react-native/ReactCommon/runtimeexecutor"
"${NODE_MODULES_DIR}/react-native/ReactCommon/yoga"
"${PYTORCH_INCLUDE_DIRS}"
"../cxx/src"
"src/main/cpp"
)
else()
file (GLOB LIBFBJNI_INCLUDE_DIR "${BUILD_DIR}/fbjni-*-headers.jar/")

target_include_directories(
${PACKAGE_NAME}
PRIVATE
"${LIBFBJNI_INCLUDE_DIR}"
"${NODE_MODULES_DIR}/react-native/React"
"${NODE_MODULES_DIR}/react-native/React/Base"
"${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/jni"
"${NODE_MODULES_DIR}/react-native/ReactAndroid/src/main/java/com/facebook/react/turbomodule/core/jni"
"${NODE_MODULES_DIR}/react-native/ReactCommon"
"${NODE_MODULES_DIR}/react-native/ReactCommon/callinvoker"
"${NODE_MODULES_DIR}/react-native/ReactCommon/jsi"
"${NODE_MODULES_DIR}/react-native/ReactCommon/runtimeexecutor"
"${PYTORCH_INCLUDE_DIRS}"
"../cxx/src"
"src/main/cpp"
)
endif()

# find libraries

Expand All @@ -90,20 +122,21 @@ find_library(
log
)

find_library(
FBJNI_LIBRARY
fbjni
PATHS ${LIBRN_DIR}
NO_CMAKE_FIND_ROOT_PATH
)

find_library(
REACT_NATIVE_JNI_LIB
reactnativejni
PATHS ${LIBRN_DIR}
NO_CMAKE_FIND_ROOT_PATH
)
if(${REACT_NATIVE_MINOR_VERSION} LESS 71)
find_library(
FBJNI_LIBRARY
fbjni
PATHS ${LIBRN_DIR}
NO_CMAKE_FIND_ROOT_PATH
)

find_library(
REACT_NATIVE_JNI_LIB
reactnativejni
PATHS ${LIBRN_DIR}
NO_CMAKE_FIND_ROOT_PATH
)
endif()

find_library(
PYTORCH_LIBRARY
Expand All @@ -112,8 +145,21 @@ find_library(
NO_CMAKE_FIND_ROOT_PATH
)

# For RN 0.66+, use the distributed libjsi.so
if(${REACT_NATIVE_MINOR_VERSION} GREATER_EQUAL 66)
if(${REACT_NATIVE_MINOR_VERSION} GREATER_EQUAL 71)
target_link_libraries(
${PACKAGE_NAME}
ReactAndroid::folly_runtime
ReactAndroid::glog
ReactAndroid::jsi
ReactAndroid::reactnativejni
fbjni::fbjni
${PYTORCH_LIBRARY}
)
elseif(${REACT_NATIVE_MINOR_VERSION} LESS 66)
# JSI lib didn't exist on RN 0.65 and before. Simply omit it.
set (JSI_LIB "")
else()
# For RN 0.66+, use the distributed libjsi.so
find_library(
JSI_LIB
jsi
Expand Down
81 changes: 54 additions & 27 deletions react-native-pytorch-core/android/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,18 @@ def safeExtGet(prop, fallback) {
android {
compileSdkVersion safeExtGet('PyTorchCore_compileSdkVersion', Integer.parseInt(appRnMinorVersion) < 68 ? 30 : 31)
buildToolsVersion safeExtGet('PyTorchCore_buildToolsVersion', Integer.parseInt(appRnMinorVersion) < 68 ? '30.0.2' : '31.0.0')

// ndkVersion getExtOrDefault('ndkVersion')
// use below because ndk 22+ version will cause `ld: error: found local symbol '__bss_start'`
// ref to https://github.com/pytorch/pytorch/issues/51020#issuecomment-1600272599
ndkVersion "21.4.7075529"

if (Integer.parseInt(appRnMinorVersion) >= 71) {
buildFeatures {
prefab true
}
}

defaultConfig {
minSdkVersion safeExtGet('PyTorchCore_minSdkVersion', 21)
targetSdkVersion safeExtGet('PyTorchCore_targetSdkVersion', 30)
Expand All @@ -67,12 +79,15 @@ android {
lintOptions {
disable 'GradleCompatible'
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
if (Integer.parseInt(appRnMinorVersion) < 71) {
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
}
externalNativeBuild {
cmake {
// version "3.22.1"
path "CMakeLists.txt"
}
}
Expand Down Expand Up @@ -106,31 +121,43 @@ repositories {

dependencies {
def pytorchLiteVersion = '1.12.2'
implementation "org.pytorch:pytorch_android_lite:${pytorchLiteVersion}"

//noinspection GradleDynamicVersion
api "com.facebook.react:react-native:+" // From node_modules

//noinspection GradleDynamicVersion
extractHeaders("com.facebook.fbjni:fbjni:+:headers")
//noinspection GradleDynamicVersion
extractJNI("com.facebook.fbjni:fbjni:+")

def nodeModules = "${rootDir}/../node_modules"

// For now we use the release version of the jni library
// we will look back on this once we figure out how to
// get the buildType duing configuration stage
def buildType = "release"
if (Integer.parseInt(appRnMinorVersion) < 71) {
implementation "org.pytorch:pytorch_android_lite:${pytorchLiteVersion}"
} else {
implementation ("org.pytorch:pytorch_android_lite:${pytorchLiteVersion}") {
exclude group:'com.facebook.fbjni', module: 'fbjni-java-only'
}
}

if (Integer.parseInt(appRnMinorVersion) < 69) {
def rnAAR = fileTree("${nodeModules}/react-native/android").matching({ it.include "**/**/*.aar" }).singleFile
extractJNI(files(rnAAR))
if (Integer.parseInt(appRnMinorVersion) < 71) {
api "com.facebook.react:react-native:+" // From node_modules
} else {
// React Native 0.69
def rnAarMatcher = "**/react-native/**/*${buildType}.aar"
def rnAAR = fileTree("${nodeModules}/react-native/android").matching({ it.include rnAarMatcher }).singleFile
extractJNI(files(rnAAR))
//noinspection GradleDynamicVersion
implementation("com.facebook.react:react-android")
}

if (Integer.parseInt(appRnMinorVersion) < 71) {
//noinspection GradleDynamicVersion
extractHeaders("com.facebook.fbjni:fbjni:+:headers")
//noinspection GradleDynamicVersion
extractJNI("com.facebook.fbjni:fbjni:+")

def nodeModules = "${rootDir}/../node_modules"

// For now we use the release version of the jni library
// we will look back on this once we figure out how to
// get the buildType duing configuration stage
def buildType = "release"

if (Integer.parseInt(appRnMinorVersion) < 69) {
def rnAAR = fileTree("${nodeModules}/react-native/android").matching({ it.include "**/**/*.aar" }).singleFile
extractJNI(files(rnAAR))
} else if (Integer.parseInt(appRnMinorVersion) < 71) {
// React Native 0.69
def rnAarMatcher = "**/react-native/**/*${buildType}.aar"
def rnAAR = fileTree("${nodeModules}/react-native/android").matching({ it.include rnAarMatcher }).singleFile
extractJNI(files(rnAAR))
}
}

extractForNativeBuild("org.pytorch:pytorch_android_lite:${pytorchLiteVersion}")
Expand Down Expand Up @@ -207,8 +234,8 @@ task extractAARForNativeBuild {
def configureCMakeTaskName = Integer.parseInt(appRnMinorVersion) < 68 ? "externalNativeBuild" : "configureCMake"
tasks.whenTaskAdded { task ->
if (task.name.contains(configureCMakeTaskName)) {
task.dependsOn(extractAARHeaders)
task.dependsOn(extractJNIFiles)
task.dependsOn(extractAARHeaders)
task.dependsOn(extractAARForNativeBuild)
}
}
Loading