Longhorn

2016-11-21T12:24:51Z

Jpaulhan:

__TOC__
==Overview==

Longhorn is a server application that allows you to execute Batch Configurations remotely on any set of input files. Batch Configurations which include pre-defined pipelines and filter configurations, can be exported from [[Rainbow]].

The distribution also includes a client library to access the Longhorn Web services.

==Download and Installation==

* '''Stable release: http://bintray.com/okapi/Distribution/Longhorn

* <del>Development release (snapshot): http://okapiframework.org/snapshots</del> Development snapshots are not currently available.

To install Longhorn:

* Unzip the distribution file on your server.
* Follow the instructions provided with the <code>readme</code> file of the distribution.
* Starting with m24, Longhorn requires Java 1.7.

==Functionality==

To process files with Longhorn these steps are required:
# Create a temporary project
# Upload a Batch Configuration file into that project
# Upload the input files into that project
# Execute the project
# Download the output files
# Delete the project

==Usage==

There are three ways to access Longhorns functionality. There is
* a REST interface,
* a Java API and
* an HTML client.

They can be used as described below.

===REST-Interface===

Longhorn can be accessed directly via HTTP methods:
;POST http://{host}/okapi-longhorn/projects/new : Creates a new temporary project and returns its URI (e.g. <code>http://localhost/okapi-longhorn/projects/1</code>)
;POST http://{host}/okapi-longhorn/projects/1/batchConfiguration : Uploads a Batch Configuration file
;POST http://{host}/okapi-longhorn/projects/1/inputFiles.zip : Adds input files as a zip archive (the zip will be extracted and the included files will be used as input files)
;PUT http://{host}/okapi-longhorn/projects/1/inputFiles/help.html : Uploads a file that will have the name 'help.html'
;GET http://{host}/okapi-longhorn/projects/1/inputFiles/help.html: Retrieve an input file that was previously added with PUT or POST
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute : Executes the Batch Configuration on the uploaded input files
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US/de-DE : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and the target language set to 'de-DE'
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US?targets=de-DE&targets=fr-FR : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and multiple target languages, 'de-DE' and 'fr-FR'
;GET http://{host}/okapi-longhorn/projects/1/outputFiles : Returns a list of the output files generated
;GET http://{host}/okapi-longhorn/projects/1/outputFiles/help.out.html : Accesses the output file 'help.out.html' directly
;GET http://{host}/okapi-longhorn/projects/1/outputFiles.zip : Returns all output files in a zip archive
;DEL http://{host}/okapi-longhorn/projects/1 : Deletes the project
;GET http://{host}/okapi-longhorn/projects : Returns a list of all projects on the server

===REST-Interface Sample code: Python===

This example works with the requests package - minidom is used to parse the XML project list.

import requests
from xml.dom import minidom

url = 'http://localhost:8080/okapi-longhorn/'

Code to create a new project

r = requests.post(url+'projects/new')
print r.text

Code to '''list''' existing projects (i.e.: to check if the project was created, and to get the ID of the last project)

r = requests.get(url+'projects/')

xmlstring = minidom.parseString(r.text)
itemlist = xmlstring.getElementsByTagName('e')
lastproject = len(itemlist)

Code to '''post''' a '''batch config file'''

batchfile = open('/home/user/batchconfig.bconf', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/batchConfiguration', files=dict(batchConfiguration=batchfile))

Code to '''put''' a string as a '''file'''

payload = "hello world!"
r = requests.put(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

Code to '''post''' a '''file'''

payload = open('/home/user/test.txt', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

===Java API===

The API is distributed as a <code>.jar</code> file in the Longhorn distribution package. You can also build it from the Okapi source code via Maven from the project <code>lib-longhorn-api</code>.

====Maven====
The API is available as a maven dependency. Add this repository to your <tt>pom.xml</tt>:
<repository>
<id>okapi-longhorn-release</id>
<name>Okapi Longhorn Release</name>
<url>http://repository-opentag.forge.cloudbees.com/release/</url>
</repository>

Along with this dependency, substituting in a valid version number (e.g, <tt>0.27</tt>):
<dependency>
<groupId>net.sf.okapi.lib</groupId>
<artifactId>okapi-lib-longhorn-api</artifactId>
<version>${okapi.version}</version>
</dependency>

====Sample Code====

LonghornService ws = new RESTService(new URI("http://localhost:9095/okapi-longhorn"));

// Create project
LonghornProject proj = ws.createProject();

// Post batch configuration
File bconfFile = new File("C:\\setup.bconf");
proj.addBatchConfiguration(bconfFile);

// Send input files

// First by single upload...
File file1 = new File("C:\\help.html");
// * in the root directory
proj.addInputFile(file1, file1.getName());
// * and in a sub-directory
proj.addInputFile(file1, "samefile/" + file1.getName());

// ...then by package upload
File inputPackage = new File("C:\\more_files.zip");
proj.addInputFilesFromZip(inputPackage);

// Execute pipeline
// Languages don't matter
proj.executePipeline();
// Languages matter
proj.executePipeline("en-US", "de-DE");

// Get output files
ArrayList<LonghornFile> outputFiles = proj.getOutputFiles();

// Does the fetching of files work?
for (LonghornFile of : outputFiles) {
InputStream is = of.openStream();
//TODO save InputStream to local file
}

// Delete project
proj.delete();

===HTML-Client===

You can create projects and upload/download files via an integrated HTML client, too. Uploading input files (and downloading output files) as a zip archive is currently not implemented for the HTML client.

[[File:longhorn_html_client.png]]

===Configuration===
Since Okapi M22 Okapi Longhorn can be build to run multiple instances on one server.
You can adjust the build so that it is possible to run multiple Longhorn instances in one JBoss application server. Therefore, the build must be called with an additional parameter:

mvn clean verify -DuseUniqueContextRoot

====Configure working directory path====
Longhorn has 2 options to configure the working directory of longhorn (sort by priority):
#system parameter "LONGHORN_WORKDIR"
#configuration file in user.home "/okapi-longhorn-configuration.xml"
If nothing is defined, the working-directory is in user.home in folder "Okapi-Longhorn-Files".
Longhorn configuration file example:

<longhorn-config>
<use-unique-working-directory>True</use-unique-working-directory>
<working-directory>D:\testData\longhorn-files</working-directory>
</longhorn-config>

====Configuration Options====

{| class="wikitable"
! option
! description
! data type
|-
| working-directory
| path of the working directory
| string
|-
| use-unique-working-directory
| if set to true the version of longhorn will be added to working directory name
e.g path/to/working/directory_M0.21
| boolean(True or False)
|}

[[Category:Longhorn]]

Okapi Framework - User contributions [en]

Longhorn