Importing from Oddmuse?

Importing from Oddmuse?

[Peter Davis]

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

I'm comparatively new to Org mode (actually, I've used it for years, but
only a small subset of its functionality). I've used Oddmuse for years to
maintain my own personal Wiki, but now I'm looking to move to Org mode.

I know there are lots of tools for exporting or publishing from Org mode to
Oddmuse, but how about the other direction? Any tools or tips for importing
a large number of Oddmuse pages into Org mode? Ideally, I'd like to keep
them as separate files, with links converted to file links, etc.

Ideas?

Thanks!

-pd


-- 
----
The Tech Curmudgeon
http://www.techcurmudgeon.com

[-- Attachment #2: Type: text/html, Size: 1009 bytes --]

Re: Importing from Oddmuse?

[Peter Davis]

[-- Attachment #1: Type: text/plain, Size: 1405 bytes --]

Just to answer my own question, I shamelessly took Alex Schroeder's 
raw.pl script and hacked it up a bit to do some conversion from Oddmuse 
markup to org-mode. The attached Perl script should run through all the 
pages in an Oddmuse Wiki and generate .org versions of them in a 
separate directory.

This is still very much a work in progress, but I think the general 
framework is useful. On thing I have to fix is the hyperlinks. Right 
now, if the Wiki page is "one two.pg", this script will generate a file 
named "one_two.org," but any links will refer to "[[file:one 
two.org][one two]]"

I concentrated on the small subset of Oddmuse markup that I'm using, but 
I think it's easily extensible.

Let me know if this is at all useful to anyone else.

-pd


On 10/25/13 10:54 AM, Peter Davis wrote:
> I'm comparatively new to Org mode (actually, I've used it for years, 
> but only a small subset of its functionality). I've used Oddmuse for 
> years to maintain my own personal Wiki, but now I'm looking to move to 
> Org mode.
>
> I know there are lots of tools for exporting or publishing from Org 
> mode to Oddmuse, but how about the other direction? Any tools or tips 
> for importing a large number of Oddmuse pages into Org mode? Ideally, 
> I'd like to keep them as separate files, with links converted to file 
> links, etc.
>


-- 
Peter Davis
The Tech Curmudgeon
www.techcurmudgeon.com


[-- Attachment #2: om2org.pl --]
[-- Type: text/x-perl-script, Size: 3893 bytes --]

#! /usr/bin/perl -w

# Copyright (C) 2005, 2007  Alex Schroeder <alex@emacswiki.org>
#
# Portions copyright (c) 2013, Peter Davis <pfd@pfdstudio.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.

sub ParseData {
  my $data = shift;
  my %result;
  while ($data =~ /(\S+?): (.*?)(?=\n[^ \t]|\Z)/sg) {
    my ($key, $value) = ($1, $2);
    $value =~ s/\n\t/\n/g;
    $result{$key} = $value;
  }
  return %result;
}

sub FixMarkUp {
    my $data = shift;
    my $orgout = "#+STARTUP: showeverything logdone\n#+options: num:nil\n\n";
    my $csvMode = 0;
    foreach (split /\n/, $data) {
	if (length($_)) {
	    s/\r//g;
	    # csv tables
	    if ($_ =~ /<csv>/) {
		$csvMode = 1;
		s/<csv>/#+ATTR_HTML: :border 2 :rules all :frame border/g;
	    } elsif ($_ =~ /^\s*$/) {
		$csvMode = 0;
	    } elsif ($csvMode) {
		s/^/|/g;
		s/,/|/g;
		s/$/|/g;
	    }
	    # hyperlinks
	    s/\[\[([^]]*)\]\]/[[file:$1.org][$1]]/g;
	    # strike through
	    s/<\/?s>/+/g;
	    # verse
	    s/:::/#+BEGIN_VERSE/g;
	    # bold and italic
	    s/'''/*/g;
	    s/''/\//g;
	    # bullet lists
	    s/^\*\*\*\*/    */g;
	    s/^\*\*\*/   */g;
	    s/^\*\*/  */g;
	    s/^\*/ */g;
	    # headers
	    s/^\=\=\=\=/****/g;
	    s/^\=\=\=/***/g;
	    s/^\=\=/**/g;
	    s/^\=/*/g;
#	    s/ \=?$//g;
	    s/ \=\=\=\=$//g;
	    s/ \=\=\=$//g;
	    s/ \=\=$//g;
	    s/ \=$//g;
	    s/^# / 1. /g;
	} else {
	    $csvMode = 0;
	}
	$orgout = $orgout . $_ . "\n";
    }
    return $orgout;
}

sub main {
  my ($regexp, $PageDir, $OrgDir) = @_;
  # include dotfiles!
  local $/ = undef;   # Read complete files
  foreach my $file (glob("$PageDir/*/*.pg $PageDir/*/.*.pg")) {
    next unless $file =~ m|/.*/(.+)\.pg$|;
    my $page = $1;
    next if $regexp && $page !~ m|$regexp|o;
    $page = $page . ".org";
    mkdir($OrgDir) or die "Cannot create $OrgDir directory: $!"
      unless -d $OrgDir;
    open(F, $file) or die "Cannot read $page file: $!";
    my $data = <F>;
    close(F);
    my $ts = (stat("$OrgDir/$page"))[9];
    my %result1 = ParseData($data);
    my $result2 = FixMarkUp($result1{text});
    if ($ts && $ts == $result1{ts}) {
      print "skipping $page because it is up to date\n" if $verbose;
    } else {
      print "writing $page because $ts != $result{ts}\n" if $verbose;
      open(F,"> $OrgDir/$page") or die "Cannot write $page org file: $!";
      # print F $result1{text};
      print F $result2;
      close(F);
      utime $result1{ts}, $result1{ts}, "$OrgDir/$page"; # touch file
    }
  }
}

use Getopt::Long;
my $regexp = undef;
my $page = 'page';
my $dir = 'org';
GetOptions ("regexp=s" => \$regexp,
	    "page=s"   => \$page,
	    "dir=s"    => \$dir,
	    "help"     => \$help);

if ($help) {
  print qq{
Usage: $0 [--regexp REGEXP] [--page DIR] [--dir DIR]

Writes the org wiki text into plain text files.

--regexp selects a subsets of pages whose names match the regular
  expression. Note that spaces have been translated to underscores.

--page designates the page directory. By default this is 'page' in the
  current directory. If you run this script in your data directory,
  the default should be fine.

--dir designates an output directory. By default this is 'org' in the
  current directory.

Example: $0 --regexp '\\.el\$' --dir elisp
}
} else {
  main ($regexp, $page, $dir);
}

Re: Importing from Oddmuse?

[Achim Gratz]

Peter Davis writes:
> 	    # hyperlinks
> 	    s/\[\[([^]]*)\]\]/[[file:$1.org][$1]]/g;

Try this to fix the links maybe:

 	    # hyperlinks
 	    s/\[\[([^]]*)\]\]/
            my $l = $1;
            $l =~ s: :_:g;
            "[[file:$l.org][$l]]"/gex;


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Importing from Oddmuse?

[Peter Davis]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]


On 10/28/13, 2:12 PM, Achim Gratz wrote:
> Peter Davis writes:
>> 	    # hyperlinks
>> 	    s/\[\[([^]]*)\]\]/[[file:$1.org][$1]]/g;
> Try this to fix the links maybe:
>
>   	    # hyperlinks
>   	    s/\[\[([^]]*)\]\]/
>              my $l = $1;
>              $l =~ s: :_:g;
>              "[[file:$l.org][$l]]"/gex;

Excellent! I modified it slightly to keep the spaces in the display string:

          # hyperlinks
          s/\[\[([^]]*)\]\]/
          my $l = $1;
          my $orig = $1;
          $l =~ s: :_:g;
          "[[file:$l.org][$orig]]"/gex;

Thanks, Achim!

-pd

-- 
----
Peter Davis
The Tech Curmudgeon
www.techcurmudgeon.com


[-- Attachment #2: Type: text/html, Size: 1762 bytes --]

Re: Importing from Oddmuse?

[Achim Gratz]

Peter Davis writes:
> Excellent! I modified it slightly to keep the spaces in the display
> string:

This is better, I'd think:

# hyperlinks
s/\[\[([^]]*)\]\]/
my $l = $1;
$l =~ s: :_:g;
"[[file:$l.org][$1]]"/gex;



Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: Importing from Oddmuse?

[Peter Davis]

On 10/28/13, 3:18 PM, Achim Gratz wrote:
> Peter Davis writes:
>> Excellent! I modified it slightly to keep the spaces in the display
>> string:
> This is better, I'd think:
>
> # hyperlinks
> s/\[\[([^]]*)\]\]/
> my $l = $1;
> $l =~ s: :_:g;
> "[[file:$l.org][$1]]"/gex;

That was the first thing I tried, and Perl complained about an undefined 
variable. I may have made a typo though.

Thanks!

Cheers,
-pd

-- 
Peter Davis
The Tech Curmudgeon
www.techcurmudgeon.com

Re: Importing from Oddmuse?

[Achim Gratz]

Peter Davis writes:
> That was the first thing I tried, and Perl complained about an
> undefined variable. I may have made a typo though.

Nope, my error.  $1 gets clobbered by the second replacement.  So you'd
want what you wrote or somewhat shorter:

# hyperlinks
s/\[\[([^]]*)\]\]/
  my ($l, $o) = ($1, $1);
  $l =~ s: :_:g;
  "[[file:$l.org][$o]]"/gex;


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Importing from Oddmuse?

[Peter Davis]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Mon, Oct 28, 2013 at 4:26 PM, Achim Gratz <Stromeko@nexgo.de> wrote:

> Peter Davis writes:
> > That was the first thing I tried, and Perl complained about an
> > undefined variable. I may have made a typo though.
>
> Nope, my error.  $1 gets clobbered by the second replacement.  So you'd
> want what you wrote or somewhat shorter:
>
> # hyperlinks
> s/\[\[([^]]*)\]\]/
>   my ($l, $o) = ($1, $1);
>   $l =~ s: :_:g;
>   "[[file:$l.org][$o]]"/gex;
>
>
Yes, that works beautifully, and is nice and concise.

Thank you!

-pd

-- 
----
The Tech Curmudgeon
http://www.techcurmudgeon.com

[-- Attachment #2: Type: text/html, Size: 1389 bytes --]

Re: Importing from Oddmuse?

[Jambunathan K]

Peter Davis <pfd@pfdstudio.com> writes:

> I've used Oddmuse for years to maintain my own personal Wiki, but now
> I'm looking to move to Org mode.

I am in the process of adding Org-mode markup support to the Oddmuse
wiki engine.  (By support I mean, only the text markup not agenda or
babel related stuff) 

I have some changes in my local work-area that is promising.

Interested people can watch out for activity in this space - 

   http://oddmuse.org/wiki/Org_Markup_Extension

One can also look at the churnings happening here:

   http://repo.or.cz/w/orgmuse.git

Re: Importing from Oddmuse?

[Peter Davis]

On Thu, Oct 31, 2013 at 02:38:48AM +0530, Jambunathan K wrote:
> Peter Davis <pfd@pfdstudio.com> writes:
> 
> > I've used Oddmuse for years to maintain my own personal Wiki, but now
> > I'm looking to move to Org mode.
> 
> I am in the process of adding Org-mode markup support to the Oddmuse

That would be great. I'd love to use the same markup everywhere.

Thanks,
-pd


> One can also look at the churnings happening here:
> 
>    http://repo.or.cz/w/orgmuse.git

Re: Importing from Oddmuse?

[Marcin Borkowski]

Dnia 2013-10-30, o godz. 18:44:53
Peter Davis <pfd@pfdstudio.com> napisał(a):

> On Thu, Oct 31, 2013 at 02:38:48AM +0530, Jambunathan K wrote:
> > Peter Davis <pfd@pfdstudio.com> writes:
> > 
> > > I've used Oddmuse for years to maintain my own personal Wiki, but
> > > now I'm looking to move to Org mode.
> > 
> > I am in the process of adding Org-mode markup support to the Oddmuse
> 
> That would be great. I'd love to use the same markup everywhere.

+1

> 
> Thanks,
> -pd
> 
> 
> > One can also look at the churnings happening here:
> > 
> >    http://repo.or.cz/w/orgmuse.git
> 



-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University