Sample Project Blog -- Improving MATHS Tools

2009-10-20 Tue Oct 20 12:22 Fixed a bug in mth2html

Yesterday I noticed that some items in a section were not being numbered.

By looking at what was numbered and what was unnumbered I noticed this
Table

Syntax	Numbered	Unnumbered
Had a inline formula	no	yes
Had no inline formula	yes	no


(Close Table)


So I looked for why a "<li>" tab was not put in when a "<span class="Inline...>" was present.

And found that I used the code like this

 		/<span/!s/...../<span class="Formula"/

to stop multiple insertions.

The fix was obvious

 		/<span class="Formula"/!s/...../<span class="Formula"/

and worked.

I cloned this into the renderer (m2h) for submissions...

Oddly a similar overgeneral regular expression was causing a different bug in the script I use to publish newer items (pubn). There I looked for ".pubn" rather than "\.pubn" and so picked up a mystype ",pubn".

Doh!

2009-09-29 Tue Sep 29 10:58 Tabs in Tables

Previously mth2html would treat a series of ASCII.HT (horizontal tabs) as a separator between table elements (<th> or <td>).

I've changed it (after it bit me) so that each single tab character in a table heading (.Table) or row (.Row) indicates a table entry. THis leads to empty entries. I have a vague memory of these not being treated well in some early browsers. [Contact] me if you have a bad experience.

After a private test period, if nothing bad happens, I'll change the script used with web pages (m2h) to0 have the same behavior.

2009-08-25 Tue Aug 25 11:39 doi DOA

It turns out that "doi:" is not a reliable protocol.... to put it mildly. I will be removing it. In place of the reccomended

 doi:10.1016/j.scico.2007.07.001

on the Science Direct web site..... it has to be

 http://dx.doi.org/10.1016/j.scico.2007.07.001

as the "href" in the HTML anchor.

I'll test this, but I'm not sure that it is a reliable feature....

It does seem to work.... and even handles ACM's DOIs.

1. DOI::acronym="Digital Object Identifier", following

   ---

   The Digital Object Identifier (DOI(R)) System is for identifying content objects in the digital environment. DOI(R) names are assigned to any entity for use on digital networks. They are used to provide current information, including where they (or information about them) can be found on the Internet. Information about a digital object may change over time, including where to find it, but its DOI name will not change.
   1. From [ http://www.doi.org/ ]

   ---

   2009-08-25 Tue Aug 25 11:28 added doi to mth2html

   Yesterday I added "doi" to the protocols that my MATHS to HTML translator handles. It was a simple change. But the ASCII browser I use to test pages doesn't handle the protocol. Safari does. I'd like to know if MSIE, Firefox, etc. do the right thing with "DOI"s. Select [ ../test.html ] to get the HTML rendering for this test case

    .Open Test

    .Source RJB

    .See ftp:ftp.csci.csusb.edu/dick/test.html

    .See doi:10.1016/j.scico.2007.07.001

    .Close

   2009-06-02 Tue Jun 2 13:35 Obvious solution

   There was an obvious solution to the problems of upgrading to a new server which didn't have the functions I needed for the PHP scripts. As Suggested be Ken our systems administrator -- I can use the old server to run the scripts.

   All I had to do was change

    comment.html:<form method=POST action="http://www.csci.csusb.edu/dick/samples/mailme.php">

    hole.html:<form method=POST action="http://www.csci.csusb.edu/dick/samples/mailme.php">

   to

    comment.html:<form method=POST action="http://ftp.csci.csusb.edu/dick/samples/mailme.php">

    hole.html:<form method=POST action="http://ftp.csci.csusb.edu/dick/samples/mailme.php">

   and the Contact and Contribute/hole pages should work.

   2009-05-30 Sat May 30 07:35 Upgrades...

   We have upgraded to a new web server for faculty. ANd this has killed the PHP scripts that use EMail on this site... [Contact]....

   It has also killed the rendering PHP scripts.... This is effectively a repeat of the next entry. Which seems to be anexample of this theorem

2. upgrade = crash.

   2009-05-06 Wed May 6 13:46 Another crash

   On Monday morning the server went down and needed to be rebuilt. The new server was up by the afternoon -- with improved security. It also returned to December 2nd 2008.

   But -- I keep a separate working copy of all files that I publish on the web. So I was able to replace all the changed files by Tuesday Morning.

   And discovered that some scripts that used PHP Mail were dead. My systems Admin got that fixed this morning and we worked on the problem of the shell scripts that came out of this project.

   Turns out that the PATH to the "bin" directory had changed with the new server. And I fixed that about 15 minutes ago. This was a "Tech" issue [ maths/T.html#PHP+unix ] from July 31st raising its ugly head again.

   2009-02-07 Sat Feb 7 08:02 Just when we got it to work...

   A young officer wrote to his parents from his place of deployment that just when they got everything to work properly somebody from home came out and reorganized everything....

   Which is how I feel about my sabbatical work. In December of this year someone got into our web server and set up what was probably a gambling site.... so the whole server had to be rebuilt and this broke all my scripts for sending contributions.

   The sysadmin was very helpful in providing work arounds and I've just installed them.

   I alsohad to find out were my "web/bin" directory had got to.

   And discover that "~dick" on the web server works differently to which I thought...

   ANd.... the our PHP doesn't like "else if".

   BUT I've fixed most of the critical scripts: home, samples, papers, maths, cs375, and cs201.

   So, in theory I'm open for contributions again.... I think I'm going back to bed as well, to recover...

   By the way, the young officer was in the Imperial Roman Army, nearly two thousand years ago. The only constant..... is change.

   2008-11-18 Tue Nov 18 14:11 Allow 3 leading spaces to work like a tab

   Here is a feature I lost.... and which is helpful when inputing MATHS into a form in some browsers. To make a formula out of a like put a tab or 3 spaces in front of it. Like
3. this
4. or this
5. for example.

   2008-10-18 Sat Oct 18 11:10 Error in fixing slashes

   I didn't know that PHP also prefixed a backslash in front of backslashes.... or more precisely I've just rediscovered this while testing "hole.php".

   So the ed shell script reads:

    	ed - "$1" <<ENDIT

    	g/\\\\'/s//'/g

    	g/\\\\"/s//"/g

    	g/\\\\\\\\/s//\\\\/g

    	w

    	q

    	ENDIT

   The octuple slashes to match what is actually a pair and the quadruple backslash to generate a single one..... was discovered thru several painful iterations:-) But now TeXie things like α don't come out as \α.

   2008-10-05 Sun Oct 5 17:10 Updated Systems and Technologies

   I've just changed the description of systems [ maths/S.html ] to date the systems as achieved.

   Also added a technique/tool [ maths/T.html#unix2dos ] for converting .mth (Unix) files into .txt (DOS) files for those servers that don't have the Gnu unix2dox command.

   2008-09-19 Fri Sep 19 13:09 Ready for Brave Beta Testers

   Go to [ ../hole.html ] and send me your thoughts.

   2008-09-19 Fri Sep 19 12:09 m2h works

   For a whiel the m2h script output nothing.... but while walking to get some tea I figured out that the web server couldn't find one of the most useful programs that I have ever developed. It is called br and it has the function of br-eaking lines along word boundaries, if possible. It was designed using a variation of Jackson Structured Programmig before ANSI C existed. The documentation is here [ br.d.html ] and you can get a copy of the C source code from [ ../tools/br.c ] , enjoy!

   Now all I have to do is make PHP create a temporary foo.mth file and pass it to fix.slashes and m2h to present to the user the HRML rendering of their submission.

   2008-09-18 Thu Sep 18 14:09 Scream of trivial triumph

   It turns out that I was working in the wrong 'bin' directory and had mistyped the permissions so that the server couldn't execute the "fix.slashes" command.

   The correct directory is obscured because PHP sees a different directory structure to a logged-in user running a shell...

   I have been here and done this before.

   2008-09-18 Thu Sep 18 13:09 Struggling with PHP--Shell

   Currently I've got PHP to run my "cookie" shell script OK, but the "fix.slashes" script is not executing. Need inspiration to see what stupid assumption I am making.

   Next step -- create a cut down version of mth2html called 'm2h` that works on a test file tester.mth.

   Then use PHP to create a tmp file ready for m2h.

   2008-09-12 Fri Sep 12 15:09 How to remove magic slashes

   PHP inserts magic slashe in front of quotation marks in incoming data in an attempt to thwart SQL injection and other evil attacks. These can also bite shell scripts -- an adde quote allows the user have any shell commands they like executes.

   However the venerable ed editor can not be tricked into executing the file it is editting, by data in that file. And so the safest(Q) technique (T) is to use a script that calls an ed command.

   Here is fix-slashes

   ---

    : fix extra slashes injected by PHP

    if [ $# -ne 1 -o "x$1" = x ]

    then

    	echo $0: Wrong number of arguments

    	exit 2

    fi

    if [ -w $1 ]

    then

    ed - $1 <<ENDIT

    g/\\\\'/s//'/g

    g/\\\\"/s//"/g

    w

    q

    ENDIT

    else

    	echo $0: file $1 not writable

    	exit 1

    fi

   ---

   2008-07-31 Thu Jul 31 16:07 Problem with PHP and UNIX resolved

   [ maths/T.html#PHP+unix ]

   2008-06-25 Wed Jun 25 10:06 Problem with PHP and Unix

   [ maths/T.html#PHP+unix ]

   2008-06-04 Wed Jun 4 06:06 Added Qs and Ts

   See [ maths/T.html ] , [ maths/Q.html ] , and [ maths/log.txt ] for details.

   Spent a week studying [Weber05] on "The Success of Open Source". Which fits well with Shirky and "Coding Horror" below.

   Consequence: confirmed that a email+reflect back system is the best next step.

   2008-05-30 Fri May 30 14:05 Many small changes

   In particular include the fact that the current bibliography of software engineering has links to more than 100 different hosts. [ maths/index.html ]

   2008-05-30 Fri May 30 12:05 Evil that men do

   Added examples: [ 001123.html ] (Coding Horror blog "Designing For Evil", May 2008). to [ maths/R.html ] (Realitites of project).

   2008-05-28 Wed May 28 18:05 Insight into how social web sites work

   Just spent the afternoon studying a wise paper on the design of technology to support conversations and how groups actually work.

   Shirky03

   1. Clay Shirky
   2. A Group Is Its Own Worst Enemy
   3. ETech (Apr 2003 ) + Published July 1, 2003 on the "Networks, Economics, and Culture" mailing list. [ group_enemy.html ]
   4. =TALK =HISTORY WWW/NET SOCIAL GROUP PEOPLE CULTURE WEB2.0 TEAM
   5. Refers to W.R. Bion: "Experiences in Groups": Groups of people become both collections of individuals and a self-coordinating entitiy. Therapeutic groups tend to: talk sex, identify external enemies, and religiously venerate something. Need for structure for groups to work. In particular rules for creating rules...
   6. Examples of BBSs, Usenet, .... where open free group is invaded.
   7. Therefore need structure in social systems.
   8. Technology (TCP/IP, WiFi, IM, Mobile phones, ...) now lets all people be online together.
   9. Example. Ito's Conference call moderated on chat with wiki for references.
   10. For a given technology -- most groups fail!
   11. Accept:
       
       1. Social and technical issues are intertwingled: they can not be separated niether does the technical drive the social. The system will have antics -- emergent properties. There will be a formal rules and informal rules.
       2. Members are not just users. There will be an onion structure. Example: reader -> anonimous coward -> named person -> moderator. The Core subgroup love and weed the garden that others wander through (and vandalize?). = Volunteer fire department.
       3. One user = one vote does not work when anybody can be a user.
       
       
   12. Things to design for
       
       1. Give members a handle -- stable local name. So they get a reputation.
       2. Make a simple way for behavior to be visible -- who is in good standing?
       3. Make it difficult to enter the core subgroups. The group is the real user!
       4. Make the communications scale. Encourage a small world structure. Many linked small groups.
       
       

   2008-05-21 Wed May 21 09:05 Tweaked the infrastructure

   [ maths/ ]

   2008-05-20 Tue May 20 19:05 Technology exploration

   This morning I discovered a difference between the 'sed' available in our labs and office machines, and the login server vs the 'sed' on our web server. My plan is to initially reuse the source code and much of that uses the UNIX Stream EDitor -- -- --(sed)
   

   The code that works outside the web server looks like this

    sed '/^\.As_is/!{s/\\TeX/\&Tau;<sub>\&Epsilon;<\/sub>\&Chi;/g

    s/\\alpha/\α/g

    ...

    }'

   and I had to "Destructure" it and introduce a goto:

    sed '/^\.As_is/bignore

    s/\\TeX/\&Tau;<sub>\&Epsilon;<\/sub>\&Chi;/g

    s/\\alpha/\α/g

    ...

    :ignore'

   Please don't tell me to use Perl.

   I'm thinking about using a variation of PArnas's Display concept where requirements, designs, the properties of machines are all shown at once, but kept separately....

   2008-05-16 Fri May 16 10:05 Infrastructure -- New directory -- samples/maths

   [ maths/ ]

   2008-05-15 Thu May 15 17:05 Blog Started -- Copying items from main blog

   I've decided that I need to track my thinking about MATHS and developing its tools separately to to my regular blog of reading. The following entries are ripped out of the normal blog.

   2008-05-13 Tue May 13 14:05 Requirments vs Technologies

   Meanwhile -- working on requirements and the available technologies for my sabbatical project, and sorting out leaks in (1) my accounting and (2) a bath room tap. Also reviewing the syntax and semantics of documentation in my MATHS language -- [ maths/notn_13_Docn_Syntax.html ] [ maths/notn_14_Docn_Semantics.html ] [ maths/notn_15_Naming_Documentn.html ] etc. .

   2008-05-12 Mon May 12 14:05 YAML -- XML considered harmful

   My favorite blog just sent me an excellent article [ 001114.html ] which says something that I have been thinking ever since XML was invented -- that it may not be the best solution for all your data encoding needs. As evidence the author gives examples of YAML [ samples/languages.html#YAML ] which are both easier to read and still 99% unambiguous.

   Like my own MATHS language it use white space to indicate structure. But it goes further by having no open+close bracketing syntax. It is all done by indentation. Much the same way CODIL did (does?).

   I'm tempted to formalize YAML into my MATHS language as a long format for complex objects:

6. CIRCLE( center=> POINT( x=>1, y=>2 ), radius => 10 )

    .Open.YAML

        center:

           x : 1

           y : 2

        radius : 10

    .Close.YAML

   But this will need some thought about current features of MATHS and how simple implementation might be. Can it be done with the UNIX standard tool kit -- sed/auk/...?

   2008-05-09 Fri May 9 17:05 Sabbatical Project -- MATHS

   My sabbatical project is not making much progress. The next post is typical of the reading that I have been doing that indicates that there may not be a market for a simple, formal notation for mathematics -- even if designed to have a lot in common with programming languages. The language exists [ maths ] and has stabilized over several years personal use. My project is to add a new feature heading in the direction of a wiki.

   In the project I was hoping to move in the direction of a wiki-style site that used the language with a lot of people contributing information, notes, samples, and so on. Yesterday I looked into a "PHP Phrasebook". It is full of warnings about the bad things can do if they supply data and some of the things you can do to avoid them. I knew about the risk of passing HTML code with "<script>" tags and have (for years) translated HTML-special symbols in MATHS into entities on HTML pages.

   What I had not thought about is that URLs can not be trusted. I'm quite paranoid about clicking on links and attachments for example in EMail. But I'm starting to wonder if I can trust links in the WikiWikiWeb and the Wikipedia any more.

   This is fairly typical of requirements... a desirable feature comes with a poison pill. It is also typical that you discover the poison pill after the project is well under way. It is also an example of a negative requirement -- something that must not be possible.

   MATHS Project Features

   1. (REQ1): Users should be able to edit MATHS web pages.
   2. (REQ2): Users must not be able to include links to illegal and/or malicious URLs.
   3. (NAT1): Some people attempt to include illegal and malicous links in my pages.
      
   4. (above)|-Anonymous users may need to have their changes vetted before inserting.
      
   5. (above)|-Untrusted users may be limited to local links.
      
   6. (above)|-There may be away to join a group of trusted users who will get full access to the system
   Note: In the above I distinguish requirements (labeled REQn) from known facts (NATn). This comes from the SCR project and Michael A Jacksons recent publications.

   Contact me if you have an thoughts.

   2008-05-05 Mon May 5 13:05 Software Reliabillity Probabillity etc

   I've made some small improvements to some pages introducing my MATHS language: [ ../maths/intro_records.html ] showing how to describe structures with optional and multiple parts. For example to state that a cat has 4 legs...
7. CAT::=Net{ cat: $ LEG ^ 4 ,...}.

   2008-03-24 Mon Mar 24 17:03 Sabbatical Next Quarter

   I've just posted my grades for the Winter quarter so I'm ready to take a break (taxes + a trip?).

   Next quarter I will be on sabbatical. This is a long tradtion in Universities. I guess it is assumed that after 7 years of teaching you will need a break to get your perspective and energy back. In CSUSB you have to apply and the aplication includes a form defining the project you plan to carry out on the sabbatical. Here is the short description

   ---

   To develop software and a web site that enable other computer scientists to experiment with and use my MATHS language. The software will enable the creation of linked, mathematical, and searchable documentation in HTML, XML, and ASCII.

   ---

   Here [ sabbatical.html ] are the details.

   I hope to start a new blog to track progress... and I have to decide where it fits this website. More later.

   Glossary

8. MATHS::= See http://csci.csusb.edu/dick/maths/, a language for semiformal documentation including ontologies, logics, conceptual models, specifications, and algorithms that I also use for weblogs, essays, lecture notes, etc. etc.

. . . . . . . . . ( end of section Sample Project Blog -- Improving MATHS Tools) <<Contents | End>>