Semantic Multimedia Annotation Tool

From KP-Lab Tools Help

Useful links
Frequently Asked Questions
Feedback form: Let us know what you think!

Contents 1 Introduction 2 SMAT scenario 3 Encoding video 4 Annotating FLV video with SMAT 5 Creation of logins and connection to SMAT 6 Upload the video 7 Structure the video and give access rights 8 Configure the annotation sequence 9 Add annotations to the video 10 Visualize annotations

[edit] Introduction

The aim of this document is to provide some material to help pedagogical partners to use SMAT. The scenario described below provides an example of video annotation by two groups of students.

[edit] SMAT scenario

Let us consider that a teacher plans to make a video annotation sequence with two groups of 2 students. They must use the following scenario:
- encode the video to the FLV format
- use SMAT to annotate the FLV video:-

• create teacher and students logins (teacher and students)
• upload the FLV video (teacher)
• structure the FLV video (teacher)
• give video access rights to students (teacher)
• configure the annotation sequence (teacher)
• annotate the video (students)
• visualize the annotations (teacher and students)

[edit] Encoding video

The current version of SMAT supports video streaming from the Adobe Flash Media Server (FMS). Due to this, only video encoded within an FLV format can be streamed. Various freeware software tools exist to perform conversion from any video format to FLV. One of them is Riva FLV encoder (http://www.clubic.com/lancer-le-telechargement-30116-0-riva-flv-encoder.html). Please upload this tool, install it on your computer and use it to encode your video. The end result will be an flv format that you will upload to the streaming server at INPT.

[edit] Annotating FLV video with SMAT

SMAT is a versatile tool for video annotation. It allows the user to establish and perform its own annotation scenario. A wide variety of scenarios are possible. This tutorial covers a basic scenario in which we have two types of users: teacher and student. The teacher plays the role of leader. He plans and organizes the video annotation sequence. Four students take the role of members and annotate the video according to the organisation established by the teacher. In this context, the teacher can visualize the annotations made by all students, whereas a student can only see her own annotations.

[edit] Creation of logins and connection to SMAT

Before you can use SMAT, you must create an account for each user. Any user can choose the role (leader or member) they want to play in a particular annotation scenario. Leaders have all privileges while members can only add and visualize annotations.

The figure below shows the creation of two students (student1 and student2) and one teacher.

In order to be identified in subsequent sessions, the user has to:

1. provide a login and password (e.g. login : student1 and password : student1)
2. submit to the data to the system by clicking the “New User” button. This will create the user and if the creation was successful, the user is automatically logged in.
   

For subsequent connections, the user must type in their login and password and click the “Login” button.


configure the annotation sequence (teacher) annotate the video (students) visualize the annotations (teacher and students) create teacher and students logins (teacher and students)
upload the FLV video (teacher) structure the FLV video (teacher) give video access rights to students (teacher) configure the annotation sequence (teacher) annotate the video (students)
visualize the annotations (teacher and students) Encoding video The current version of SMAT supports video streaming from the Adobe Flash Media Server (FMS). Due to this, only video encoded within an FLV format can be streamed. Various freeware software tools exist to perform conversion from any video format to FLV. One of them is Riva FLV encoder (http://www.clubic.com/lancer-le-telechargement-30116-0-riva-flv-encoder.html). Please upload this tool, install it on your computer and use it to encode your video. The end result will be an flv format that you will upload to the streaming server at INPT.

[edit] Upload the video

Any user can upload his/her FLV video to the server. This step is required (for this version) in order to be able to annotate the video. Future versions will be able to connect to videos from any streaming server. The user who uploads the video is called the owner of the video. Owners can give access rights to others users to view and annotate their videos. As such, only authorized users can video materials. The default situation that allows all users to view and annotate is not yet implemented. In our scenario, the teacher uploads the video, structures it and gives rights to the different students. To perform this, the teacher proceeds as follow:

1. Click the “upload” button
2. Select the video into the local file system
3. Click “open”

The upload process starts and might take few minutes. Long videos might take quite a long time to upload. At the end of the process, the new video will appear on the left-hand list. This list shows all content items for which the user has access (read, annotate). The user can browse this list. Each content item can consist of various parts (for videos these are called clips). Selecting a clip will preview the video on the right hand side of the screen.

When the selection is done, the user must “choose next activity” to proceed with SMAT. The following window will appear. Here the user chooses what he/she wants to do: structure a video (cut it into clips, segments…), configure an annotation session (organise the activity by creating sequences, phases, defining ontologies, roles…), annotate a video (create actual annotations), visualize annotations (view annotations of a group, perform some statistical analysis…)

Here the user must first choose his/her role. Leaders can select any of the 5 activities. Members can only annotate and visualize. When selection is done, the user clicks “Start activity” to proceed.

[edit] Structure the video and give access rights

As mentioned above, the user can structure a video into clips. A clip is a container that can group various video segments from the same video (currently only one segment is allowed). A video segment is a sequence of contiguous images. As such, the user can define different interest parts, cutting off long unnecessary parts. The user might define different clips to focus on different issues in the content. To structure the video, the user should have chosen a leader role and started the “Structure the Multimedia Content” activity.

The displayed screen composes of three zones:

• Zone 1: list of clips defined into the content item. You can add (by clicking “Add Clip” button), rename (by double clicking the clip), or delete (by clicking “Delete Clip” button) a clip. When a new clip is created, it consists of a single segment that covers the whole video. Its name is as follows: n1-n2, where n1 and n2 are numbers of seconds. n1 is the relative time where the segment starts. n2 is the end time.
• Zone 2: the user can use controls in this zone to adjust the video clip start and end time. This is done by moving the corresponding markers to the desired positions. The video display area shows the video images corresponding to the marker position.
• Zone 3: in this zone, the user selects the other users allowed to view this clip. Selected users are highlighted by holding the control key (on the keyboard) and clicking their names. They are unselected by clicking again. The owner is selected by default. When the list is complete, the user should click “save the list” to submit the rights.

In our scenario, the leader aims to create two groups of students. The first one (student1 and student2) will add annotations into the first part of the video (until 250 s) and the second one (student3 and student4) into the second part of the video (between 250 s and the end of the video). To create such a scenario, the leader should:

• Create and rename the two clips
• Adjust the size of each clip
• Gives access rights to the students for each clip

To modify the size of the segment for a clip, the leader should:

1. Select the segment in the Zone1
2. Click the button
3. Move the sliders on the time line
4. Click a second time the button

When this is finished, the user clicks “return to the activity choice” in order to proceed with the tool.

[edit] Configure the annotation sequence

Usually, when the clips are created, the next step will be the configuration of the annotation session. For this, the user clicks on “configure the annotation session”. The following screen appears. In the area entitled “List of clips”, the user can view all existing clips. In each clip, he can view annotation sequences and phases. An annotation sequence is just a list of phases. And a phase is a sort of directory where the group will store their annotations. A sequence may consist of a number of phases (for examples one for each day). These two elements are used by the users as they feel useful to structure their annotations. Subsequent restructuring is possible by creating, a posteriori, phases and sequences and copying or removing annotations. Any newly created clip comes with one sequence and one phase by default. The user can add other sequences and phases.

You can add, rename and delete sequences (or phases) by selecting the operation into the dropdown menu . When you select a clip, you can see the users authorized to participate to annotate it. In the current version, the tabs “domain ontologies” and ‘phase Members” are not yet implemented. In our scenario, the leader decides to create two sequences of annotation for the first group and two phases for the second one. This produces the following screen:

When this configuration is done, the user returns to the activity screen by clicking on the button “return to the activity choice”.

[edit] Add annotations to the video

In SMAT, any authorized user can add annotations (leaders or members). In our example scenario, only students add annotations. For this purpose, they must log in .

As stated above, student1 and student2 will add annotations into the first clip; and student3 and student4 will annotate the second clip. This has been stated into the configuration of the scenario.

In our scenario, the students must select the member role to add annotations to the video. If they do not do so, they would not be able to view the annotation scenario set up by the teacher (even if they are leaders).

Each student must select the phase in which he/she would like to contribute. Then they click the tab “Manage Annotation Session”.

Into the tab “Manage Annotation Session”, the student can add semantic annotation or informal annotation. Currently, we can only use the paediatric ontology for semantic annotations. Future versions will include different ontologies. It is possible to switch between textual and graphical representations of the ontology.

To add an annotation to the video, the user follows the following procedure:

1. Read and control the video using these buttons
2. When the desired image is reached, click on the pause button
3. Click on the button to switch into the annotation mode
4. Select the type of annotation (semantic or informal)
5. Select the value (or type the text in the case of informal annotation)
6. Add optionally a graphical fragment ( or ) onto the image
7. Click again on the button to terminate the annotation mode
8. The annotation appears in the table on the bottom of the screen

A student logged in SMAT with member role, sees only her own annotations. Users logged in with leader role are able to see all annotations made by all members of the group working in the same phase. In our scenario, there are two ontologies, one for the leaders and one for the members. Users logged in with leader role will have the leader paediatric ontology. User logged in with member role have the team member ontology.

It is possible to select an annotation into the table by clicking on it. The video would then display the corresponding image and the graphical symbol (circle or square) if any has been added when the annotation was created.

When you click on the, the annotation is deleted. When the annotation activity is done, the user clicks the “return to the activity choice” button to return to the activity choice screen.

[edit] Visualize annotations

At any moment, leaders can view the group annotations. As an example, the same users can log in with leader role to view the annotations. In our scenario, when all students have finished adding annotations, the annotations are visualized by the teacher. For this purpose he/she should :

1. Log in with the teacher login
2. Choose the clip to analyse
3. Choose the leader role and the “visualise annotations” option
4. Select the phase to analyse and click on the tab “visualize annotations” or “statistics”

The hard copies of screens below show the views corresponding to this sequence.

The next screen appears. Select the phase to analyse.

When you click on the tab “visualize annotations”, all the annotations made into the phase are displayed. You can now click on one of the lines into the table to go to the annotation. It is possible to sort annotation by any of the columns, by just clicking on the header of the column (Author, date, ontology, concept, value…).

The statistics view offers the possibility to display statistical results about the occurrence of annotations. However, the current version has limited analysis options.

1. Select the type of data to analyse (userRole, ontology, concept or value)
2. Click on the button “Submit”
3. See the table result into the tab “table of statistic” or a graphical result into the tab “global stats Pie chart”.