As of now I am publishing the updated code of the standalone version, which has partially been re-written in two ways: for one it now comprises of the same functionality as the online version, and for another the coding structure has been reorganized so I can keep the two versions in unison easier. This means that any update or change made to the software will be applied to both versions: the online version will make use of the update whereas the standalone version will be published open source under the same license with the updated code.

You can download the updated code here.

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Let me quickly list the options which the new standalone version offers (i.e. the differences to the original publication), which are about user customizations:

• $OUTFILE_TYPE = “TXT”
  defines whether the type of the output should be in text (TXT) or comma-separated values (CSV). Please note that the file extension you define for $OUTFILE as part of the filename is not taken into consideration; so e.g. if you want to output a CSV file, setting the outfile name to “outfile.csv” is not enough, you need to specifically set $OUTFILE_TYPE to CSV.
• $NEWLINE = “\n”
  defines the format of the system dependent newline. Use \r\n on Windows and \n on Unix/Linux and Mac.
• default file opening mode is now “overwrite” ( $f_handle = fopen( $OUTFILE, “w” ); ). If you would like to append extracted text message to the output file rather than overwrite the file use “a” instead of “w“. Note that the line now moved towards the end of the file.
• Change format of output of single text messages
  In case of TXT you need to change $OUTSTRING .= “$time$senderror$NEWLINE$address$NEWLINE$text$NEWLINE$NEWLINE$NEWLINE”; to your liking and in case of CSV change $OUTSTRING .= “$address,$time,\”" . str_replace( “\”", “\”\”", $text) . “\”$NEWLINE”;.
  Make sure you only change the format string and leave the .= intact .

If you are a new user to this and don’t know what to do about the user specific CONFIGURATION section or how to run the script, please refer to the original blog post, which explains in detail how the standalone version works.

As always you can contact me with any remarks you would like to bring to my attention.

Because I’m currently having more fun to do some programming in PHP than to rewrite the original script in Java (most people will have a JVM and/or JRE installed), I created a new webpage that uses the original script.

Click: SMS / text message extraction from iPhone

The usage is simple: select the SQLite database file, make customizations if desired (otherwise default values will be used) and click “Extract”. You might have to wait a little while depending on the size of your storage file. After the file has been (automatically) uploaded and messages have been extracted, you will be offered a download prompt for the result file.
See the above page for further details.

Compared to other online solutions out there, this version (as well as the original script) has more ways of tailoring the output to your own needs and the usage is utterly free. However, so far you cannot yet change the arrangement of phone number, date and message in the new online version; if you want to do that, you will still have to use the original work (which was released under CC BY 3.0).

Please leave comments about the usage into this blog post, so I can improve the page and script if necessary.

Cheers

UPDATE (21/11/2011): Correct CSV export was added.

UPDATE (05/01/2012): Now supports iOS 5 container.

For a while I’ve been checking out various programs for extracting and converting text messages from the iPhone SMS storage SQLite database. For some reason they seem to be either for purchase or not doing their job well.
When I found out that all they are doing is pretty much just some simple SQL query and putting the result into various output formats, I decided to quickly write this myself, while keeping it simple and open.

Download

The choice of a programming language wasn’t easy taking into account that most iPhone users will be on Windows and the extractor should not require major alterations to the system (and doing that without the solution itself being bloated).
Even though the task can be easily done in various languages such as Perl, Python, PHP, C/C++ or Java, I decided to use PHP, because the program can stay very simple and can run with a minimal version of PHP. In addition many users with a Unix/Linux machine available will have PHP installed anyway and can also easily use the program.
That being said we can move on to the program details. The PHP script needs the iPhone SMS storage SQLite database file as input and will then convert all text messages and append them to a file in a certain format.
So far the script is configurable to a certain extent. Naturally the name of the input and output file can be modified. Furthermore you can decide whether you want to extract just incoming or outgoing texts or all of them. Last but not least a date range can be given, from which to extract the messages. See further down (or code) for details about customization.
At this point in time, it is not (yet) possible to limit extraction to a single contact. Even though it might be a common use case for many users, however the format of recipient/sender may vary (at least in my storage file), so it might be hard to determine which number formats are from the same contact.

The usage of the extractor is as follows:

1. First you will need the SMS storage container file (it is usually named 3d0d7e5fb2ce288813306e4d4636395e047a3d28 and can be found in the iTunes backup folder, which can be found e.g. on Windows7 in C:\Users\[username]\AppData\Roaming\Apple Computer\MobileSync\Backup\[hexstring] or get it directly from your iPhone: /var/mobile/Library/SMS/sms.db )
2. If you’re on Unix/Linux, make sure you have PHP5 (along with libs for sqlite) installed. If you’re on Windows, you can either use the official PHP installer [alternative: http://windows.php.net/download/] (just run the installer) or use my minimized packaged version of PHP (just unpack to a location of your desire)
3. Download the SMS Extractor for iPhone (extract_sms.php_.txt), rename it to extract_sms.php and use a text editor of your desire to make configuration changes (see below and/or follow the instructions and explanations in the code)
4. If you used the PHP installer, navigate to the folder containing extract_sms.php and double-click on it (make sure you placed the SMS storage container file in the same directory and modified the input configuration)
5. If you used my minimized version, place extract_sms.php in the same folder you unpacked the PHP archive; use Start, Run…, “cmd.exe” and navigate to the correct folder / if you’re on Unix/Linux, navigate to the folder you downloaded extract_sms.php)
6. (make sure you placed the SMS storage container file in the same directory and modified the input configuration) then run the script typing “php extract_sms.php”
7. Unless you changed the output file, you will find a file “sms.txt” in the same folder

Possible customization can be done in the following ways (look for “user specific CONFIGURATION” in extract_sms.php ):

• $DATABASE = “sms.db”
  change this to the appropriate filename if your SMS storage container file is named differently
• $TIME_FORMAT = “d.m.Y H:i:s”
  the internal text message timestamp is the number of seconds since Unix Epoch, so in order to make this human readable, provide the time format of your desire (see http://www.php.net/manual/en/function.date.php for details)
• $FILTER = “ALL”
  set here whether you would like to limit extraction to only incoming text messages (“IN“), or only outgoing messages (“OUT“) or no limit (“ALL“)
• $OUTFILE = “./sms.txt”
  name for the output file, you can change this to any name with or without file extension
• date_default_timezone_set(‘Europe/Berlin’)
  if you want to use a time range (see next item), you need to specify your timezone (which supposedly equals the timezone your text messages got timestamped). See http://www.php.net/manual/en/timezones.php for available timezones
• $RANGE_BEGIN = mktime( 22, 45, 00, 11, 9, 2009 ) and/or $RANGE_END = mktime( 22, 55, 00, 11, 9, 2009 )
  if you want to use a time range, you can configure either a start or an end of range or even both. The order of comma separated digits in parentheses is: hour, min, sec, mon, day, year. Also, you need to uncomment (i.e. remove the leading //) the line of the variable(s) you want to use
• Replace file instead of appending ( $f_handle = fopen( $OUTFILE, “a” ); )
  the default behavior is to open the output file in append mode in order to avoid accidental overwrite. If you would like replace the output file instead of appending the extracted text messages, put “w” instead of “a“
• Change output format ( fwrite( $f_handle, “$time$senderror\n$address\n$text\n\n\n” ); )
  if you would like to change the format in which the script is putting your extracted text message information, you will need to change this line appropriately. However, I cannot provide a manual how to change it to your liking at this moment; you will either know what you’re doing or have to google and trial ‘n error

Please feel free to comment about the usage or drop me an email at blog {at} haraldkraft(.)de. It’s fine with me to continue development and add new (simple) features, and definitely fix bugs I might have there in PHP; just let me know .

Also, please do use my script for your adaptions; I am releasing this under CC.

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Following the official description how to subscribe to a remote calendar did add the calendar successfully, but showed a yellow warning sign with the caption

The calendar foobar is momentarily not available

This message actually points to a connection problem, network wise or maybe concerning the webserver. Even though I had done everything according to the description, it didn’t help.
For a long time, this was the status quo – surprisingly the calendar worked from time to time, without any explanation.

However, trying to open an exported .ics file with Lightning/Sunbird pointed me into another direction, because it still claimed to be momentarily not available, even though no network or other connection was involved.
Honestly, I should have tried it before, but this was the first time I opened Thunderbird’s Error Console to see a more descriptive warning message:

Warning: There has been an error reading data for calendar: foobar. Error code: CAL_UTF8_DECODING_FAILED. Description: An error occurred while decoding an iCalendar (ics) file as UTF-8. Check that the file, including symbols and accented letters, is encoded using the UTF-8 character encoding.

and if any Germans have this problem:

Warnung: Fehler beim Lesen von Daten für Kalender: foobar. Fehlercode: CAL_UTF8_DECODING_FAILED. Beschreibung: Ein Fehler ist beim Dekodieren einer iCalendar(ics)-Datei als UTF-8 aufgetreten. Überprüfen Sie, dass die Datei, inklusive Symbolen und akzentierten Buchstaben, mit dem UTF-8-Zeichensatz kodiert ist.

So, obviously it was an encoding problem. I tried to save the exported ICS file in several encodings and could open it with Sunbird/Lightning if it was properly encoded in UTF-8.
The message in Thunderbird’s error console finally made it easy to google the problem (as opposed to the warning caption telling me that the calendar was momentarily not available). People seemed to have this very problem when they used special characters in their WebCalendar. And in fact, as I removed the special characters I was using in some event descriptions, loading the calendar in Lightning worked just fine.

One solution for the problem is not to use any special characters, the other one is just a workaround until they fix this issue (which is still commented as a TODO in the code btw ): This WebCalendar SourceForge Bugtracker comment tried to comment out a few lines in the WebCalendar source code. The final code of my “webcalendar-dir”/includes/classes/WebCalendar.class now looks like this (around line 870 in v1.2.0):

// if (extension_loaded('mbstring')) {
//   $mb_lang = strtok($lang, '-');
//   if (mb_language($mb_lang) && mb_internal_encoding(translate('charset'))) {
//      $enable_mbstring = true;
//   } else {
        $enable_mbstring = false;
//   }
// }

Please do comment your experiences.

<Directory /foo/bar>
Script PUT /some.php
</Directory>

And so I did. But when I tried to reload Apache’s configuration (/etc/init.d/apache2 reload), Apache complained with the following message:

Invalid command ‘Script’, perhaps misspelled or defined by a module not included in the server configuration
failed!

At this point in time, Google didn’t have any results for the error string, and this is why I am actually writing this blog entry.

Searching for Apache2′s directive Script got me to this Apache doc where you can see that the directive Script is part of the module mod_actions.
As already mentioned in the error message, a module was not included, now I knew that it would probably be the mod_actions module. Enabling the module by symlinking it from mods-enabled to mods-available made the error message disappear and everything working just fine:

ln -s /etc/apache2/mods-available/actions.load /etc/apache2/mods-enabled/actions.load
ln -s /etc/apache2/mods-available/actions.conf /etc/apache2/mods-enabled/actions.conf

Hope this helps to offer a quick solution for the error message.

Btw, while looking for fixes, I stumpled across this page, which offers further information about how to use Apache’s Script directive.

[[ OLD:
Before I can / want to release it under some free software license, I would appreciate it if some of you PHP nerds could review the code. I am mainly concerned about security issues rather than formatting and coding style. Since I am no pro in PHP coding, I will probably have done some things quite clumsily.

So please point out security problems as well as other major issues. (For example, if anybody knows how to stop parsing after max items were dealt which, please let me know ).
]]

[UPDATE 07/08/2009]
(HTTP) Links are now made clickable.
[/UPDATE]

[UPDATE 15/12/2009]
The script was released under the Creative Commons Attribution 3.0 Unported License today.
Download the PHP code at http://blog.haraldkraft.de/wp-content/uploads/2009/12/parsetwitter.php_.txt and feel free to reuse and tweak my script (along with mentioning my name as suggested in the license information in the code).
[/UPDATE]

Email me at blog {at} haraldkraft(.)de (or comment)

This work is licensed under a Creative Commons Attribution 3.0 Unported License.