Effacer les filtres
Effacer les filtres

Info

Cette question est clôturée. Rouvrir pour modifier ou répondre.

How to solve "The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information."

1 vue (au cours des 30 derniers jours)
Jingyu Ru
Jingyu Ru le 15 Août 2015
Clôturé : Jingyu Ru le 16 Août 2015
Hadoop version 1.2.1 Matlab version 2015a
Linux ubuntu 14.
At first, I try to run them at a cluster (one master and two slaves). I can run the example 'wordcount'in Hadoop. And it is success to read the data from the HDFS through the Matlab.
But when I try to run the example in Matlab 'Run mapreduce on a Hadoop Cluster',I failed.
It shows that the Map 0% and Reduce 0%.
Error using mapreduce (line 100)
The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information.
Error in run_mapreduce_on_a_hadoop (line 24)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,...
There is my Matlab codes
setenv('HADOOP_HOME','/usr/local/hadoop');
cluster = parallel.cluster.Hadoop;
cluster.HadoopProperties('mapred.job.tracker') = 'ubuntu:50031';
cluster.HadoopProperties('fs.default.name') = 'hdfs://ubuntu:8020';
outputFolder = '/home/rjy/logs/hadooplog';
mr = mapreducer(cluster);
ds = datastore('airlinesmall.csv','TreatAsMissing','NA',... 'SelectedVariableNames','ArrDelay','ReadSize',1000);
preview(ds)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,... 'OutputFolder',outputFolder)
I don't know what it means in this logs as follows,
2015-08-14 22:41:40,981 INFO org.apache.hadoop.mapred.JobInProgress: Choosing a failed task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
'attempt_201508142238_0001_m_000000_3' to tip task_201508142238_0001_m_000000, for tracker
'tracker_slave1:localhost/127.0.0.1:41664'
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201508142238_0001_m_000000_2'
2015-08-14 22:41:42,855 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201508142238_0001_m_000000_3:
com.mathworks.toolbox.parallel.hadoop.MatlabStartupFailureException: parallel:mapreduce:HadoopTaskCouldNotFindMatlab
Caused by: java.lang.ClassNotFoundException: com.mathworks.toolbox.distcomp.util.Version
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
How to solve this problem? Thanks a lot.
I attach my Hadoop logs and Matlab logs.

Réponses (0)

Cette question est clôturée.

Produits

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by