SCHEMENAUER AND THE XOR GATE

SCHEMENAUER AND THE XOR GATE

IMPLEMENTING ANN IN PYTHON

I was searching for Artificial Neural Networks (ANN) implementation in Python. I came across the following;

1. FANN - C library with python bindings
2. PyBrain
3. NeuroLab
4. PyNN 
5. BPNN - Not a library, solitary script by Neil Schemenauer

THE XOR PROBLEM

The XOR problem has some history in the evolution of ANN methods. The XOR function is not linearly separable and cannot be realised using only one layer of ANN.

TINKERING WITH SCHEMENAUER'S CODE

Schemenauer's code has default training values for a 2 input XOR gate.

Schemenauer recommends using of a (2,2,1) network (viz. a network with two input, two hidden, and one output nodes) and the output is very much as desired, in the limits of errors of the ANN.  

XOR Output for a (2,2,1) Back Propogation Neural Network;

([0, 0], '==', [0.025608579041218795])

([0, 1], '==', [0.98184578447794768])

([1, 0], '==', [0.98170742564066216])

([1, 1], '==', [-0.021030064439813451])

However, playing around with the number of hidden layers has interesting results,

XOR Output for a (2,1,1) Back Propagation Neural Network; 

([0, 0], '==', [0.0020536886211772179])
([0, 1], '==', [0.68437587415369783])
([1, 0], '==', [0.68413753288547252])
([1, 1], '==', [0.6856616998850974])

The output of (2,1,1) clearly confirms the XOR problem !

Increasing the number of hidden layers indiscriminately, leads to anomalous output.

As an example, XOR Output for a (2,25,1) Back Propagation Neural Network;

([0, 0], '==', [0.99999643777993841])
([0, 1], '==', [0.99999911082329096])
([1, 0], '==', [0.99999280130316026])
([1, 1], '==', [0.99999824824488848])

Anomalous behaviour comes into play from about 12 hidden nodes.


REFERENCES
(1) An introduction to neural networks

A VERY SIMPLE CHATBOX IN PYTHON

A VERY SIMPLE CHATBOX IN PYTHON

A naive chatbot program. No parsing, no cleverness, just a training file and output.

It first trains itself on a text and then later uses the data from that training to generate responses to the interlocutor's input. The training process creates a dictionary where each key is a word and the value is a list of all the words that follow that word sequentially anywhere in the training text. If a word features more than once in this list then that reflects and it is more likely to be chosen by the bot, no need for probabilistic stuff just do it with a list.

The bot chooses a random word from your input and generates a response by choosing another random word that has been seen to be a successor to its held word. It then repeats the process by finding a successor to that word in turn and carrying on iteratively until it thinks it's said enough. It reaches that conclusion by stopping at a word that was prior to a punctuation mark in the training text. It then returns to input mode again to let you respond, and so on.

It isn't very realistic but I hereby challenge anyone to do better in 71 lines of code !! This is a great challenge for any budding Pythonists, and I just wish I could open the challenge to a wider audience than the small number of visitors I get to this blog. To code a bot that is always guaranteed to be grammatical must surely be closer to several hundred lines, I simplified hugely by just trying to think of the simplest rule to give the computer a mere stab at having something to say.

Its responses are rather impressionistic to say the least ! Also you have to put what you say in single quotes.

I used War and Peace for my "corpus" which took a couple of hours for the training run, use a shorter file if you are impatient...

here is the trainer

#lukebot-trainer.py
import pickle
b=open('war&peace.txt')
text=[]
for line in b:
for word in line.split():
text.append (word)
b.close()
textset=list(set(text))
follow={}
for l in range(len(textset)):
working=[]
check=textset[l]
for w in range(len(text)-1):
if check==text[w] and text[w][-1] not in '(),.?!':
working.append(str(text[w+1]))
follow[check]=working
a=open('lexicon-luke','wb')
pickle.dump(follow,a,2)
a.close()

here is the bot

#lukebot.py
import pickle,random
a=open('lexicon-dict','rb')
successorlist=pickle.load(a)
a.close()
def nextword(a):
if a in successorlist:
return random.choice(successorlist[a])
else:
return 'the'
speech=''
while speech!='quit':
speech=input('>')
s=random.choice(speech.split())
response=''
while True:
neword=nextword(s)
response+=' '+neword
s=neword
if neword[-1] in ',?!.':
break
print response

You tend to get an uncanny feeling when it says something that seems partially to make sense. I think this is a universal syndrome with chatbots and shows us a lot about our expectations and hopes for machine intelligence. The tendency to attribute some kind of surreal cosmic wisdom or irony to the responses is common.

TALKING TO TOLSTOY ! ...... ;)

Sample output:( is this like talking to Tolstoy? Not really but still...)


>>>
>'hello'
the human affairs he thought.
>'oh those affairs were great'
God,
>'what -surprised?'
do you value but she is what had the new country,
>'i do value but i do not blame the war on her'
so alarming them with those around with particular chair by all those who caressed me.
>'i never get anyone caressing me!'
allowed to talk about in a really so.
>'oh do not be prudish I am allowed to mention it'
should she suddenly heard of a duty for reason was handed him very hot.
>'you like these women a lot to talk about them so much'
advantage if your father?
>'my father approves of my romantic side'
love my son's having obeyed in the causes historic events that circle they didn't you know a force producing two Emperors with a certain prince.
>'oh now it is princes is it'
God,
>'do you believe in Him?'
the insistence of incongruities giving you married at his squadron--that is,
>' i don't understand you sir'
>'i do not understand you sir'
the bridge which,
>'i crossed'
himself,
>'ok ok... well 71 lines still is not bad'
the work at Smolensk?
>'yes'
or that of War.
>'no, war'
was gradually that millions of married in which is Dolokhov?


REFERENCES

(1) Pythomism - Luke's website

JYTHON : FIRST CONTACT !

JYTHON, MARRIAGE OF JAVA AND PYTHON

Trying my first shot at Jython, using java swing in python-like interactive interface !

CHECKING FOR SQUARES !

AN ELEGANT CODE IN PYTHON !

This program confirms if a number is square or not !

A SIMPLE EXAMPLE WITH PYCHECKER

BEGINNING TESTING !

Trying out this tool for python testing .... pychecker on program e.py,

The testing software identifies that sum 'shadows builtin' , thus changing name of the function from sum to summation.

.... rectifies the Warnings !

REFERNCES

(1) Pychecker
(2) Beginning Python : From Novice to Professional

ZETA IN SCIPY

PROBLEMS WITH ZETA FUNCTION IN SCIPY

Dabbling with zeta in scipy gave various unsatisfactory results. Few very obvious results of zeta function are ;

(1) For negative even integers the function is zero

(2) zeta(0) = -1/2

(3) zeta(-1) = -1/12

(4) zeta(1/2) = -1.46035450880.....

(5) zeta(1) = infinity

(6) zeta(2) = 1.6449340.....

Trying it on scipy,

The probable reason for this inconsistent behaviour and issues for zeta values below 2 and improbable results for negative numbers is; the module is probably structured abinitio from zeta function

whilst it may be a better idea to structure the zeta function using the functional equation.

It may also help to have a special provision for zeta(1/2)

REFERENCE
(1) Zeta Scipy reference

SINE INTEGRAL IN SCIPY

THE SINE INTEGRAL

The Sine Integral is a very important function in Physics, Astronomy, Electrodynamics, Mathematical Physics,Optics and Signal Processing.

A fundamental result in the sine integral is;

This result is analytically proven using contour integrals concept from complex theory.


TRYING IT IN MATLAB



MATLAB gives excellent results, particularly for the special case of (0-inf )it gives correct value.

TRYING IT IN SCIPY

In Scipy the sine integral (and the cosine integral) is via (si,ci) = sici function. It yields excellent values for numbers, however for infinity it yields nan (not a number). This should probably be corrected with an exception in the sici module.

It is worth noting that for sufficiently high values (which tend to infinity) the desired result of 1.57.... ( = pi/2) is obtained, which confirms the numerical evaluation is correct

REFERENCES
(1) sici
(2) sine integral

RAISING IT TO THE POWER OF ......

LAWS OF EXPONENT

In the laws of exponents, a number can never be raised to an exponent to yield negative values. Only using complex exponents can negative values be obtained.


Trying the same formulation in python, it is worth noting that the formulation fails for a = 1 hence a special case output for a = 1.


Some sample output is ;


Similar treatments in MATLAB is also fruitful



The visible change is that iota in MATLAB it is i, while in Python it is j.

BIZARRE BIZARRE PYTHON

IS THAT SOME WITCH CRAFT ?

Python .... maybe the one of the best programming languages has just gone crazy !

In the interpreter mode I got these crazy results trying to get numbers starting with zero...


Only on referring to Hetland that I got to know that the interpreter does an OCTAL !

REFERENCES
(1) Hetland

SQUARE ROOT OF IOTA

SQUARE ROOT OF COMPLEX NUMBERS

i, the square root of -1 the fundamental complex number. Working out the square-root of i;

ON MATLAB

Trying it on Matlab

Fig 1. Matlab 1

Fig 2. Matlab 2

Matlab gives very precise result both by 'power of 0.5' and 'sqrt function'.

USING CMATH

Using cmath module in Python;

Fig 3. cmath 1

Fig 4. cmath 2

cmath also gives wonderful results, however it is worth noting that the real and complex parts are different in the last 2 digits ( 0.70710678118654757 in the real part while 0.70710678118654746 in the complex part); which should not be so as they both represent the same number !

USING SCIPY

Using Scipy, scientific and numerical module in python

Fig 5. Using Scipy

Similar results to that of cmath.

SQUARING THE ROOT !

Squaring the square root often confirms to the accuracy and resolution of the software.

Fig 6. Squaring the root in Matlab

Fig 7. Squaring the root in cmath

Fig 8. Squaring the root in scipy

SOME OBSERVATIONS

Matlab on squaring the root, gives precise results

cmath and scipy on squaring the root gives precise results for the complex part but odd results for the real part (2.2204460492503131e-16 for scipy and -2.2204460492503131e-16 for cmath).

Using (1j)**0.5 and sqrt(1j) in scipy yields different results in real parts (2.2204460492503131e-16 for (1j)**0.5 and -2.2204460492503131e-16 for sqrt(1j)).

For developing scipy there should be a sense of consistency with cmath and the resolution (digits in the answer) should be controlled at the discretion of the user( It really looks sleek in Matlab). Further it looks odd and conveys a sense of inconsistency if the complex part tallies completely with the expected result while the real part has an inconsistency.

EULER'S GAMMA !

EULER'S GAMMA

Once again ! .... we meet Leonhard Euler ... a constant named after him. Euler-Mascheroni constant which runs as .... 0.57721 … called 'gamma' ,denoted by the Greek alphabet 'gamma' and is one of the important constants of mathematics.

From an abinitio, 'gamma' is defined as;

IN SCIPY

Trying it out in scipy yields a very accurate gamma.....

Fig 1. gamma in scipy

Should gamma be build into scipy as pi and e ?

Fig 2. pi and e in scipy

IN MATLAB

Trying it in MATLAB

MATLAB recognises the integral as a special integral ! ...... with a vpa, the value is obtained.

Gamma and other mathematical constants should be build into Scipy and Scipy should be intelligent enough to identify these expressions and integrals.

REFERENCES
(1) Murray Spiegel

AN EXOTIC INTEGRAL IN SCIPY

ADVENTURES IN SCIPY

Scipy is a module in python which allows for mathematical and scientific functions and tools. Trying to evaluate an exotic integral , using contour integration and complex analysis it can be shown that ;

Trying out the integral in scipy, the function is introduced using lambda and the scipy.integrate.quad is used over 0 to infinity to obtain the results. The result comes up with a warning on infinite recursions and a recommendation to use a special-purpose integrator and the numeric value is 1.5708678849453777, which is with 0.0015587759422623915 of the correct value (~0.025% accurate).

Fig 1. The integral in scipy

REFERENCES
(1) Scipy
(2) Scipy mini anthology

I AM FAMOUS !

GOING PLACES ..... AGAIN !

Now, I am famous ! ...... my game Pygame Toss has been published in famouswhy ..... I would guess it is the simplicity of the game than its achievements , that makes it 'famous' !

Fig 1. I am famous !

REFERENCES
(1) Pygame Toss

VISUAL PYTHON

VPYTHON - FIRST PROGRAM !

Trying out visual python (version 5.12) came rather easy.


My first program was mere 2 lines and I could do a fair deal with that.

REFERENCES

(1) VPython intro
(2) Some great examples
(3) Wiki
(4) Vpython in teaching Physics

AN ODE TO 'pi'

3.141592653589793......

The ratio of circumference to diameter, may be no other number has intrigued and troubled mathematicians any more. The Bible puts pi as 3 ..... while many fanatics have spend the prime of their lives computing the 'little' that lies beyond 3 ... Ludolph van Ceulen from Leiden is sure worth a mention .... and he took the 35 digits of his computation to his tombstone after his death in 1610 ..... the exoticism of pi doesn't end here ..... and Feynman point is another interesting aspect of the unending digits of pi....

It must be appreciated that pi is not just another constant .....and is far from the likes of physical constants as G,h and c ..... and is also distinctly different from root 2, gamma, e and iota .... the physical constants are structured by the physical theory and are found to vary with time..... while e, iota, gamma etc can be said to be product of our chosen number system and the bias of our prevalent mathematical structure ....... while, pi is engraved in mother nature ..... it is ubiquitous and omnipresent .... pi enunciates why every circle mimics every other circle .... and so is true for every sphere..... Various physical theories as electrostatics, fluid-dynamics and gravitation have confirmed the presence of pi in their formulation , i.e: Stokes Equation, Gauss's Law, Kepler's law .....

The only other constant which may compete for similar prominence is phi , though phi is more subtle and not really as often visible as pi .... well..... I must stop with these rhetorics ... and get to business ...... 2 python recursions (1) Ramanujan's formulation (2) Wallis Product.

(1) Ramanujan's formulation

One of the most exotic and 'very fast converging' series for computing pi was given by Ramanujan...

A corresponding python program is ....

Fig 1. piramanujan.py

The program gives very accurate value of pi (3.14159265381), however the limit of recursion is reached in about 20 terms ...

(2) Wallis Product

An evaluation of pi in a 'product' form of an infinite series was given by English mathematician John Wallis.

The python program is ....

Fig 2. piwallis.py

The limit of recursion is at about 995 terms .

Though life can be made much easier ... away from these recursions by importing pi from math .....

Fig 2. pisimple.py