[Big Data] Visualize huge amounts of geo data using WebGL-Globe

For my Master Thesis I’m evaluating several interesting frameworks to visualize Big Data in the browser. Next to the fantastic D3.js I found WebGL quite helpful even if it is not as far developed as its 2D predecessors. Nevertheless, Google shows some impressive examples of what is possible with WebGL on their site chrome-Experiments.com. To give you a first impression on what can be done with those examples I want to show a screenshot I took that visualizes German students who study in a foreign country.
GermanStudentsInForeignCountries

Source code is not more than ~50 lines so that it can be adapted to your personal needs easily.

[Java] 100 common words in German, English and French

To exclude common words from word and tag clouds and determine them in text mining procedures it is neccessary to determine them beforehand. The following Java class will help you to find them in the languages German, English and French. Thanks to the University of Leipzig who created the collection.

Update: I added some common words to the german list that I found doing my research and removed and removed some duplicates.

package de.jofre.commonwords;
 
public class CommonWords {
 
	// By Jonas Freiknecht
 
	// Source: http://wortschatz.uni-leipzig.de/
 
	// There also exists a web service from the University of Leipzig
	// to determine common words:
	// http://wortschatz.uni-leipzig.de/axis/servlet/ServiceOverviewServlet
 
	public static boolean contains(String _str, String[] _list) {
		for (int i = 0; i < _list.length; i++) {
			if (_list[i].equalsIgnoreCase(_str)) {
				return true;
			}
		}
		return false;
	}
 
		public final static String[] COMMON_WORDS_GERMAN = { "der", "die", "und",
			"in", "den", "von", "zu", "das", "mit", "sich", "des", "auf",
			"für", "ist", "im", "dem", "nicht", "ein", "für", "eine", "als",
			"auch", "es", "an", "werden", "aus", "er", "hat", "daß", "sie",
			"nach", "wird", "bei", "einer", "du", "um", "am", "sind", "noch",
			"wie", "einem", "über", "einen", "ob", "so", "dessen", "zum", "war",
			"haben", "nur", "oder", "aber", "vor", "zur", "bis", "mehr",
			"durch", "man", "sein", "wurde", "sei", "RT", "bin", "hatte",
			"kann", "gegen", "vom", "können", "schon", "wenn", "habe", "seine",
			"Mark", "ihre", "dann", "unter", "wir", "soll", "ich", "eines",
			"ins", "Jahr", "zwei", "Jahren", "diese", "dieser", "wieder",
			"keine", "willst", "seiner", "worden", "Und", "will", "zwischen",
			"extra", "immer", "Millionen", "Ein", "was", "sagte", "ihr",
			"jetzt", "kennen", "sagen", "armer", "arme", "gerne", "kenne",
			"meine", "hoffe", "sehen", "achso", "reicht", "dabei", 
			"gehst", "alles", "selbst", "neuen", "neue", "liebe",
			"feiern", "letzte", "macht", "könnte", "keiner", "glaub", "glaube",
			"gehen", "euren", "passt", "passe", "passen", "findet",
			"eigentlich", "reden", "machen", "liebt", "halbe", "dieses",
			"finde", "sitze", "machen", "halbe", "sonst", "heute", "brauch",
			"drauf", "total", "meint", "denkt", "lässt", "hätte",
			"damals", "lange", "dachte", "wirst", "hören", "kennt",
			"bitte", "treffen", "würde", "fängt", "länger", "könnt",
			"sitzt"};
 
	public final static String[] COMMON_WORDS_ENGLISH = { "the", "of", "to",
			"and", "a", "in", "for", "is", "The", "that", "on", "said", "with",
			"be", "was", "by", "as", "are", "at", "from", "it", "has", "an",
			"have", "will", "or", "its", "he", "not", "were", "which", "this",
			"but", "can", "more", "his", "been", "would", "about", "their",
			"also", "they", "million", "had", "than", "up", "who", "In", "one",
			"you", "new", "A", "I", "other", "year", "all", "two", "S", "But",
			"It", "company", "into", "U", "Mr.", "system", "some", "when",
			"out", "last", "only", "after", "first", "time", "says", "He",
			"years", "market", "no", "over", "we", "could", "if", "people",
			"percent", "such", "This", "most", "use", "because", "any", "data",
			"there", "them", "government", "may", "software", "so", "New",
			"now", "many" };
 
	public final static String[] COMMON_WORDS_FRENCH = { "de", "la", "le",
			"et", "les", "des", "en", "un", "du", "une", "que", "est", "pour",
			"qui", "dans", "a", "par", "plus", "pas", "au", "sur", "ne", "se",
			"Le", "ce", "il", "sont", "La", "Les", "ou", "avec", "son", "Il",
			"aux", "d'un", "En", "cette", "d'une", "ont", "ses", "mais",
			"comme", "on", "tout", "nous", "sa", "Mais", "fait", "été",
			"aussi", "leur", "bien", "peut", "ces", "y", "deux", "A", "ans",
			"l", "encore", "n'est", "marché", "d", "Pour", "donc", "cours",
			"qu'il", "moins", "sans", "C'est", "Et", "si", "entre", "Un", "Ce",
			"faire", "elle", "c'est", "peu", "vous", "Une", "prix", "On",
			"dont", "lui", "également", "Dans", "effet", "pays", "cas", "De",
			"millions", "Belgique", "BEF", "mois", "leurs", "taux", "années",
			"temps", "groupe" };
 
}

[Gamestudio] BmapGS – Library for bitmap manipulations

For the TUST project I wrote a DLL for some advanced bitmap operations based on the cImg library. It contains for example functions like:

  • Blur
  • Resize
  • Mirror
  • Erode
  • Sharpen
  • Draw line, arrow, spline, triangle, rectangle, border, text

Have fun with it!

[Gamestudio] TUST – A new template library

Some friends and I started a new template library for the game development suite Gamestudio called TUST. This library is supposed to be easy to use for beginners to enable Gamestudio to be more user friendly as it was in its younger days. Furthermore, the Gamestudio community posts hundreds of great code snippets but nobody ever composed them into one package. Currently, we are working on the basics like path finding, effects, shaders and small demos.

If anybody is interested in joining us (yes, you too, students 😉 ) he/she can contact me. We need every help we can get so that we can show an eary state on the AckCon 2013 (video1, video2) in June in Bielefeld!

To show the progress we made, I want to post some pictures of the finished, customizable, procedurally generated menu system I wrote in the last days.

Hadoop 1.0.4 auf Ubuntu Server 12.04.2 installieren

Die Installation von Hadoop ist nicht ganz trivial und leider auch nicht allzugut dokumentiert. Allerdings existiert ein sehr gutes Tutorial von Michael Noll in dem dieser erklärt wie man einen einfachen Single-Node-Cluster aufsetzt. Dieses habe ich mehr oder weniger ins Deutsche übersetzt und stelle es hier zur Verfügung.

 

[Gamestudio] Leap Motion DLL

Today I finished a simple DLL to connect Gamestudio to the Leap Motion. Headers and DLL can be downloaded here. The plugin works with the Leap Motion SDK 0.7.6.

Download

Since I don’t know about the license, you have to download the SDK yourself and put the Leap.dll in the same directory as this plugin.

The device has to be enabled by calling leap_init() and can then be used, for instance, to get all hands (get_hand_count()) recognized by the device. The video below shows a quickly hacked demo.

[Leap Motion] Setting up a C++ project with Visual Studio 2010

Yesterday, my Leap Motion Developer Device arrived and I started imeadiately to try the diverse SDKs. Setting up a Java project is simple because the official resource side provides an easy but great tutorial. A C++ project is some more difficult but fairly uncomplicated, as well, when you know how to configure include and library paths. Follow these easy steps to set up a project:

  1. Create a Win32 project via File -> New Project…
  2. Right click the project and select Properties
  3. Under Configuration Properties -> VC++ Directories -> Include directories add a link to [LeapSDK]\include
  4. Under Configuration Properties -> VC++ Directories -> Library directories add a link to [LeapSDK]\lib\x86
  5. Add Leap.dll (Or Leapd.dll in debug mode) under Linker -> Input.
  6. Include at least leap.h in your project and follow the official C++ tutorial to begin your first project.

Have fun! 🙂

Developing custom JSF 2.0 components

In this tutorial I want to show you how to create a classic JSF 2.0 component in pure Java (Classic != Composite Component). Furthermore, you learn how to package it to an own JSF taglib (meaning: as a JAR) so that you it can be distribute easily.

 

HDFS Explorer – Managing the Hadoop file system on a remote machine

Working with hadoop means working with the Hadoop File System (HDFS). Therefore, it is mandatory to read, write and delete files via command line. That can be quite difficult and exhausting when you are not familiar with the common unix and hadoop commands. To handle this task, I wrote a small application that is able to work with an HDFS running on Ubuntu.

So far, the application is able to:

  • Read the HDFS in a treeview
  • Upload / Download files to/from the local machine
  • Delete files on the HDFS
  • Create directories on the HDFS

When there is a need (and if I get enough good feedback 😉 ), I’ll add session management for several file systems as well as the function to start MapReduce jobs from the application (as it can be seen in the lower group box).

A download is about to follow soon!

Eine einfache Logging-Klasse in C++

Ich schreibe gerade an einigen DLLs für Gamestudio und hatte da das Problem, dass es nicht einfach ist diese zu Debuggen. Dafür habe ich hier eine einfache Logging-Klasse geschrieben die als Singleton agiert, er muss also nicht explizit initialisiert sondern einfach nur aufgerufen werden. Das geht so:

Logger::getInstance()->log("Skins: %i", 1);

Es ist also möglich, beliebig lange Argumentelisten zu verarbeiten. Das Log wird in der Datei “Log.txt” im Verzeichnis der Anwendung gespeichert.

Den Quelltext findet ihr hinter dem More-Tag. Viel Spaß damit!

Continue reading “Eine einfache Logging-Klasse in C++”