You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
* Updated the Scala-Readme with upto-date information
* Updated the header
* Removed redundant build status
* Minor formatting changes
* Addressed the PR feedback
* Added section on Scala training APIs
* Removed mention of deprecated Model API
It brings flexible and efficient GPU/CPU computing and state-of-art deep learning to JVM.
4
+
The MXNet Scala/Java Package brings flexible and efficient GPU/CPU computing and state-of-art deep learning to JVM.
9
5
10
6
- It enables you to write seamless tensor/matrix computation with multiple GPUs
11
7
in Scala, Java and other languages built on JVM.
12
8
- It also enables you to construct and customize the state-of-art deep learning models in JVM languages,
13
9
and apply them to tasks such as image classification and data science challenges.
10
+
- The Scala/Java Inferece APIs provides an easy out of the box solution for loading pre-trained MXNet models and running inference on them.
14
11
15
-
Install
12
+
Pre-Built Maven Packages
16
13
------------
17
-
18
-
Technically, all you need is the `mxnet-full_2.11-{arch}-{xpu}-{version}.jar` in your classpath.
19
-
It will automatically extract the native library to a tempfile and load it.
20
-
You can find the pre-built jar file in [here](https://search.maven.org/search?q=g:org.apache.mxnet)
21
-
and also our nightly build package [here](https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~)
22
14
23
-
Currently we provide `linux-x86_64-gpu`, `linux-x86_64-cpu` and `osx-x86_64-cpu`. Support for Windows will come soon.
24
-
Use the following dependency in maven, change the artifactId according to your own architecture, e.g., `mxnet-full_2.11-osx-x86_64-cpu` for OSX (and cpu-only).
15
+
### Stable ###
16
+
17
+
The MXNet Scala/Java packages can be easily included in your Maven managed project.
18
+
The stable jar files for the packages are available on the [MXNet Maven Package Repository](https://search.maven.org/search?q=g:org.apache.mxnet)
19
+
Currently we provide packages for Linux (Ubuntu 16.04) (CPU and GPU) and macOS (CPU only). Stable packages for Windows and CentOS will come soon. For now, if you have a CentOS machine, follow the ```Build From Source``` section below.
20
+
21
+
To add MXNet Scala/Java package to your project, add the dependency as shown below corresponding to your platform, under the ```dependencies``` tag in your project's ```pom.xml``` :
**Note:**```<version>[1.3.1,)<\version>``` indicates that we will fetch packages with version 1.3.1 or higher. This will always ensure that the pom.xml is able to fetch the latest and greatest jar files from Maven.
60
+
61
+
### Nightly ###
62
+
63
+
Apart from these, the nightly builds representing the bleeding edge development on Scala/Java packages are also available on the [MXNet Maven Nexus Package Repository](https://repository.apache.org/#nexus-search;gav~org.apache.mxnet~~~~).
64
+
Currently we provide nightly packages for Linux (CPU and GPU) and MacOS (CPU only). The Linux nightly jar files also work on CentOS. Nightly packages for Windows will come soon.
65
+
66
+
Add the following ```repository``` to your project's ```pom.xml``` file :
If you have some native libraries conflict with the ones in the provided 'full' jar (e.g., you use openblas instead of atlas), this is a recommended way.
45
-
Refer to the next section for how to build it from the very source.
114
+
**Note:**```<version>[1.5.0,)<\version>``` indicates that we will fetch packages with version 1.5.0 or higher. This will always ensure that the pom.xml is able to fetch the latest and greatest jar files from Maven Snapshot repository.
46
115
47
-
Build
116
+
Build From Source
48
117
------------
49
118
50
-
Checkout the [Installation Guide](http://mxnet.incubator.apache.org/install/index.html) contains instructions to install mxnet.
51
-
Then you can compile the Scala Package by
119
+
Checkout the [Installation Guide](http://mxnet.incubator.apache.org/install/index.html) contains instructions to install mxnet package and build it from source.
120
+
If you have built MXNet from source and are looking to setup Scala from that point, you may simply run the following from the MXNet source root:
52
121
53
122
```bash
54
123
make scalapkg
55
124
```
56
125
57
-
(Optional) run unit/integration tests by
126
+
You can also run the unit tests and integration tests on the Scala Package by :
58
127
59
128
```bash
60
129
make scalaunittest
61
130
make scalaintegrationtest
62
131
```
63
132
64
-
Or run a subset of unit tests by, e.g.,
133
+
Or run a subset of unit tests, for e.g.,
65
134
66
135
```bash
67
136
make SCALA_TEST_ARGS=-Dsuites=org.apache.mxnet.NDArraySuite scalaunittest
@@ -70,123 +139,38 @@ make SCALA_TEST_ARGS=-Dsuites=org.apache.mxnet.NDArraySuite scalaunittest
70
139
If everything goes well, you will find jars for `assembly`, `core` and `example` modules.
71
140
Also it produces the native library in `native/{your-architecture}/target`, which you can use to cooperate with the `core` module.
72
141
73
-
Once you've downloaded and unpacked MNIST dataset to `./data/`, run the training example by
- To set up the Scala Project using IntelliJ IDE on macOS follow the instructions [here](https://mxnet.incubator.apache.org/tutorials/scala/mxnet_scala_on_intellij.html).
145
+
- Several examples on using the Scala APIs are provided in the [Scala Examples Folder](https://github.com/apache/incubator-mxnet/tree/master/scala-package/examples/)
84
146
85
-
If you've compiled with `USE_DIST_KVSTORE` enabled, the python tools in `mxnet/tracker` can be used to launch distributed training.
86
-
The following command runs the above example using 2 worker nodes (and 2 server nodes) in local. Refer to [Distributed Training](http://mxnet.incubator.apache.org/how_to/multi_devices.html) for more details.
147
+
Scala Training APIs
148
+
-------
149
+
- Module API :
150
+
[The Module API](https://mxnet.incubator.apache.org/api/scala/module.html) provides an intermediate and high-level interface for performing computation with neural networks in MXNet. Modules provide high-level APIs for training, predicting, and evaluating.
To run training over multiple GPUs and multiple hosts, one can use the [KVStore API](https://mxnet.incubator.apache.org/api/scala/kvstore.html).
99
154
100
-
Change the arguments and have fun!
155
+
- IO/Data Loading :
156
+
MXNet Scala provides APIs for preparing data to feed as an input to models. Check out [Data Loading API](https://mxnet.incubator.apache.org/api/scala/io.html) for more info.
157
+
158
+
Other available Scala APIs for training can be found [here](https://mxnet.incubator.apache.org/api/scala/index.html).
159
+
101
160
102
-
Usage
161
+
Scala Inference APIs
103
162
-------
104
-
Here is a Scala example of what training a simple 3-layer multilayer perceptron on MNIST looks like. You can download the MNIST dataset using [get_mnist_data script](https://github.com/apache/incubator-mxnet/blob/master/scala-package/core/scripts/get_mnist_data.sh).
The [Scala Inference APIs](https://mxnet.incubator.apache.org/api/scala/infer.html) provide an easy, out of the box solution to load a pre-trained MXNet model and run inference on it. The Inference APIs are present in the [Infer Package](https://github.com/apache/incubator-mxnet/tree/master/scala-package/infer) under the MXNet Scala Package repository, while the documentation for the Infer API is available [here](https://mxnet.incubator.apache.org/api/scala/docs/index.html#org.apache.mxnet.infer.package).
149
164
150
-
Predict using the model in the following way:
151
-
152
-
```scala
153
-
valprobArrays= model.predict(valDataIter)
154
-
// in this case, we do not have multiple outputs
155
-
require(probArrays.length ==1)
156
-
valprob= probArrays(0)
157
-
158
-
// get real labels
159
-
importscala.collection.mutable.ListBuffer
160
-
valDataIter.reset()
161
-
vallabels=ListBuffer.empty[NDArray]
162
-
while (valDataIter.hasNext) {
163
-
valevalData= valDataIter.next()
164
-
labels += evalData.label(0).copy()
165
-
}
166
-
valy=NDArray.concatenate(labels)
167
-
168
-
// get predicted labels
169
-
valpy=NDArray.argmax_channel(prob)
170
-
require(y.shape == py.shape)
171
-
172
-
// calculate accuracy
173
-
varnumCorrect=0
174
-
varnumInst=0
175
-
for ((labelElem, predElem) <- y.toArray zip py.toArray) {
176
-
if (labelElem == predElem) {
177
-
numCorrect +=1
178
-
}
179
-
numInst +=1
180
-
}
181
-
valacc= numCorrect.toFloat / numInst
182
-
println(s"Final accuracy = $acc")
183
-
```
165
+
Java Inference APIs
166
+
-------
167
+
The [Java Inference APIs](http://mxnet.incubator.apache.org/api/java/index.html) also provide an easy, out of the box solution to load a pre-trained MXNet model and run inference on it. The Inference APIs are present in the [Infer Package](https://github.com/apache/incubator-mxnet/tree/master/scala-package/infer/src/main/scala/org/apache/mxnet/infer/javaapi) under the MXNet Scala Package repository, while the documentation for the Infer API is available [here](https://mxnet.incubator.apache.org/api/java/docs/index.html#org.apache.mxnet.infer.package).
168
+
More APIs will be added to the Java Inference APIs soon.
184
169
185
-
Release
170
+
JVM Memory Management
186
171
-------
187
-
- Version 0.1.1, March 24, 2016.
188
-
- Bug fix for MAE & MSE metrics.
189
-
- Version 0.1.0, March 22, 2016.
172
+
The Scala/Java APIs also provide an automated resource management system, thus making it easy to manage the native memory footprint without any degradation in performance.
173
+
More details about JVM Memory Management are available [here](https://github.com/apache/incubator-mxnet/blob/master/scala-package/memory-management.md).
0 commit comments