A filtergraph for FMJ
GSoC Student: Martin Harvan - Slovakia
Primary Mentor: Ken Larson - USA
Project Requirements
First term:
- Implement simple version of adapted A* algorithm for finding codec path between two formats
- Write suit of tests (done by comparing results of A* with results of breadth-first algorithm) that would test correctness of construction of filtergraph from fmj’s and jmf’s codecs (and also by comparing to jmf’s results)
Second term:
- Improve the implementation by
- Adding negotiation of partialy specified formats (or possibility to build filtergraph out of partially specified formats and then negotiate once we found the shortest one),
- Correctly handling “special” codecs (image scaler, resampler)
- Ensuring performance (in terms of time) is satisfactory.
- Create a branch version of SIP Communicator that can handle all codecs supported by JMF
Project Description:
Inside SIP Communicator we have a Media Service that we use for capturing audio and video, streaming media over the net, and receiving RTP streams.
Currently the implementation of this Media Service uses the Java Media Framework (JMF) from SUN. Yet JMF has been abandoned by SUN for more than five years ago so now we�re trying to move to something newer and better: the Freedom for Media in Java (FMJ framework).
This is one of two projects that we will be mentoring in a joint effort with the FMJ community. Read on for more details:
One of the most critical parts of a multimedia infrastructure is how it connects components together such that the data flows through to perform the desired playback or processing function. Using Microsoft’s terminology, this is a “filter graph” - a graph of connected nodes which include essential tasks like parsing, demultiplexing, transcoding, multiplexing, rendering. Given a specific multimedia file or source, such as an audio/video file or an RTP input, and a desired function, such as playback (rendering), or processing to another format, a graph must be constructed from the available parsers, demultiplexers, codecs, etc. Once this graph is constructed correctly, the data must flow properly through it. FMJ has a filter graph construction algorithm, which uses the metadata about installed JMF-style plugins (codecs, etc) to construct a graph. There are a number of cases where this graph construction either takes an inordinate amount of time (considering too many possibilities), or yields a suboptimal graph with unnecessary nodes/format changes. The challenging task here will be to find a better way to construct these graphs, perhaps by studying the architecture of other successful projects like gstreamer or ffmpeg.
References:
Freedom for Media in Java
http://fmj-sf.net
The Java Media Framework
http://java.sun.com/jmf
Other Jitsi GSoC Projects
http://gsoc.jitsi.org
Jitsi Developer Documentation
http://www.jitsi.org/index.php/Documentation/DeveloperDocumentation
The official Jitsi website
http://www.jitsi.org