hadoosh

KO | EN

gitlite — search

#python #go #android #machine-learning #rust #linux #docker #macos #nodejs #java #awesome #vue #mcp #open-source

(★ 8)

Hadoop Interactive Shell

hadoosh 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

HadooSh
=======

Ever wanted to grep a file on HDFS and store the results locally?
Now you can, with one simple command in HadooSh:

```bash
cbeavers > cd books/
books > cat Ulysses-part-00000 | grep Mulligan >l localOut.txt
```

HadooSh is an interactive shell for HDFS built on top of JLine to offer
tab completion of commands and paths under Hadoop. It supports piping
to local system commands, and both local and remote output. There are
many bugs to be found, so please play nicely with it.

Currently supported operations:
-------------------------------
 - cd [dir]
 - pwd
 - head [numLines]
 - all FsShell commands (hadoop fs), such as
     - cat [files]
     - mv \<src\> \<dst\>
     - rm  [files]
 - avrocat (prints the first ten records from avro file)
 - local (to execute one of the above commands on the local FS)
 - support for piping to local commands
 - use ">" to run command output to HDFS filesystem
 - use ">l" to run command output to local filesystem
 - all JobClient commands (hadoop job) are accessible to by typing "job" after the prompt first
 - runjar \<localjar\> ...  (hadoop jar)
 - tlog (-job \<jobid\>
         |-dir \<jobOutputDir\>)
        [-taskpattern taskglob] [-hostpattern hostglob]
    
    ```bash
     # Show all logs for a job:
     tlog -job job_201306131712_0004

# Show all mapper logs for a teragen job:
     tlog -dir /user/gera/tgen -taskpattern *_m_*

# grep logs for job tasks run on certain nodes
     tlog -job job_201306131712_0004 -hostpattern *.rack.company.com | grep needle
     ```

Planned future actions:
-----------------------
 - should probably put some limits on file sizes

Known bugs:
-----------
- piped commands that contain quotes such as

```bash
cat file | cut -d" " -f 1
```

Build HadooSh:
--------------
```bash
git clone https://github.com/gerashegalov/hadoosh.git
cd hadoosh
mvn package
```
This builds HadooSh against Apache dependencies

You can also use build HadooSh against MapR artifacts:
```bash
mvn package -Pmapr
```

Executing HadooSh:
------------------
To use HadooSh, just copy the included jar to your Hadoop cluster's
gateway, make sure you've kinit'd if necessary, and run the following:

```bash
wget https://github.com/cbeav/hadoosh/raw/master/HadooSh.jar
hadoop jar HadooSh.jar HadooSh
```

To run HadooSh with Maven:
```bash
# symlink hadoop conf
ln -s ${HADOOP_HOME}/conf
mvn clean compile exec:exec -Dexec.executable=java -Dexec.args="-classpath %classpath HadooSh"
```

Enjoy.

**Authors:** Chris Beavers, Paul Hobbs, and Gera Shegalov

hadoosh

hadoosh 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

hadoosh

hadoosh 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README