|
Java example source code file (SparkDataUtils.java)
The SparkDataUtils.java Java example source codepackage org.deeplearning4j.spark.util; import org.apache.spark.api.java.JavaRDD; import org.apache.spark.api.java.JavaSparkContext; import org.canova.api.records.reader.RecordReader; import org.deeplearning4j.spark.canova.RecordReaderFunction; import org.nd4j.linalg.dataset.DataSet; /** * Created by Alex on 15/06/2016. */ public class SparkDataUtils { private JavaRDD<DataSet> loadFromTextFile(String path, int labelIndex, int numClasses, RecordReader recordReader, JavaSparkContext sc){ JavaRDD<String> lines = sc.textFile(path); // gotta map this to a Matrix/INDArray return lines.map(new RecordReaderFunction(recordReader, labelIndex, numClasses)); } } Other Java examples (source code examples)Here is a short list of links related to this Java SparkDataUtils.java source code file: |
... this post is sponsored by my books ... | |
#1 New Release! |
FP Best Seller |
Copyright 1998-2021 Alvin Alexander, alvinalexander.com
All Rights Reserved.
A percentage of advertising revenue from
pages under the /java/jwarehouse
URI on this website is
paid back to open source projects.