Blog moved to http://www.andrevdm.com/

Tuesday 15 December 2015

CentralConfig

Untitled Document.md

CentralConfig - Multi-machine configuration manager

CentralConfig is available on NuGet, source at https://github.com/andrevdm/CentralConfig

Overview

CentralConfig is a replacement for the built in .net ConfigurationManager. It makes managing multi-machine configurations simple while, from a usability point of view, looking very similar to ConfigurationManager.

Features

  • All config must be able to be stored in version control
  • A way to have config centralised. I.e. you should not have to update the same setting on multiple machines.
  • Support for multiple environments (dev, build, test, prod etc)
  • A sane and simple way of having default values. You should not have to specify the same settings for each machine if a default will work.
  • Override a setting for a particular machine. e.g. one machine in an environment must have a slightly different config.
  • Simple and light weight
  • Support for simple values as well as more complex object types
  • A full replacement for ConfigManager

What about the .net config transformer

The .net transformer does a reasonable job but did not quite do everything I required. The examples below should show some of the differences

Using CentralConfig

Install “CentralConfig” from nuget Unless you have created a custom persistor use the mongo settings persistor Set the central config mongo settings Setup DI

app.settings for the mong persistor

   <appSettings>
     <add key="MongoDB.Server" value="mongodb://localhost:27017" />
     <add key="CentralConfig.MongoDatabase" value="TestConfig" />
   </appSettings>

DI setup

   ObjectFactory.Configure( x =>
   {
       x.For<IEnvironment>().Use<SystemEnvironment>();
       x.For<IConfigPersistor>().Use<MongoDbConfigPersistor>();
   } );

Configure settings in Mongo

   {
       "key" : "TestSetting",
       "machine" : null,
       "value" : "some value"
   }

Use the setting from code

   //ConfigManager.AppSettings["TestSetting"]
   Console.WriteLine( "test: {0}", ConfigManager.AppSettings["TestSetting"] );

Different Persistors

I’ve always used the mongo persistor but you can easily create new persistors as required. To do so implement the IPersistor interface

    public interface IConfigPersistor
    {
        string ReadAppSetting( string key, string machine, long version );
        string ReadAppSetting( string key, string machine );
        string ReadAppSetting( string key, long version );
        string ReadAppSetting( string key );

        TResult GetSection<TResult>( string sectionName, string machine, long version );
        TResult GetSection<TResult>( string sectionName, string machine );
        TResult GetSection<TResult>( string sectionName, long version );
        TResult GetSection<TResult>( string sectionName );
    }

Order of matching

When you request a value, CentralConfig will try find the most specific value it can. So it will first look for the setting match both the key and the current machine name, if that does not match then it looks for a value matching on key only

   {
       "key" : "TestSetting",
       "machine" : null,
       "value" : "some value"
   },
   {
       "key" : "TestSetting",
       "machine" : "MYPC",
       "value" : "another value"
   }

Given the mongo settings above when you request “TestSetting” you will get “another value” if your machine name is MYPC or “some value otherwise”

Monday 12 January 2015

Capturing multiple channels of digital data on a two channel digital oscilloscope

In my last post I showed how I parsed the CSV data to do some basic logic analysis. The problem is that I have a two channel scope and need to capture three channels for SPI (i.e. clock + MOSI + MISO).

As it turns out this can be done without too much fuss by using a digital to analogue (DAC) "resistor ladder". The two options I looked at were the R-2R ladder and the simpler binary weighted ladder. For two levels the binary weighted DAC is the simplest, it just needs two resistors (one double the other one's resistance).

Here is the circuit diagram

So one input goes through the 1k resistor and the other through a 2k resistor and you measure the output. With an DAC with only two levels there is tolerance for some noise on the digital lines.

I'm working at ~3 volts, so the levels I'm going to see are
Voltage D0 D1
0v 0 0
1V 0 1
2V 1 0
3V 1 1


You can clearly see the three levels on the scope when measuring Vout



I kept the clock separate on channel 2 so that I could trigger on it. Here is the clock + data



With only two channels going through the DAC you could reconstruct the two digital data channels by eye. That is not much fun though. So I modified my F# script from the previous post to split the data into two values and then do the logical analysis and print out two channels of data. This proved very useful when using a SPI device that returns data (e.g. SD card). Here is the full F# script also available as a gist.

Logic level parsing Rigol oscilloscope CSV data with F#

I was recently debugging a SPI communication but had no logic analyser. Fortunately my oscilloscope (rigol DS1052e) can export a capture as CSV (I assume that most digital oscilloscopes can export to CSV too).
Parsing the CSV is trivial, getting a logic level from it is also fairly easy. I realise that this is terribly quick & dirty but it worked 100% for what I needed. Here is the high level process
  1. Ignore timings - SPI is clocked so actual timing is irrelevant for this analysis
  2. Parse X and Y channels
  3. Convert clock channel voltage into logic level 1 or 0
  4. Discard all rows apart from rows where the clock goes high
  5. Convert data channel to logic level
  6. Group into bytes
  7. Display hex value

As I said, very simple but it worked well and I was able to see exactly what was going on. Obviously this only works for synchronous (clocked) protocols like SPI/I2C but that is all I needed.

Here is the full F# script (also see the gist)

Monday 5 January 2015

Accurate clock with any frequency oscillator (e.g. embedded PIC projects)

When I created an electric timer using a PIC one of the things I needed was an accurate clock. The PIC I was using did not have a built in real-time clock (RTC) and I did not want to use an external one. After some searching I found two popular suggestions to create an accurate clock
  1. Use a 32.768kHz crystal which divides down perfectly to give you seconds etc
  2. Use any frequency oscillator and manually set the value of the timer register on each interrupt. I.e. to skip some number of cycles
I did not like either of these solutions
  1. 32.768kHz is ridiculously slow compared to the 20-40MHz the PICs can run at
  2. Manually setting the timer register takes some number of clock cycles and this will affect the timing. Working out how many clock cycles exactly sounds like an error prone and rather boring task
Fortunately I found this article "A very versatile Zero Cumulative Error timing system with PIC source code" @ http://www.romanblack.com/one_sec.htm. It shows a really simple way to get an accurate clock, i.e. no accumulating error,using any oscillator frequency. Take a look at the article. It has PIC source code and a nice explanation.
Its a brilliantly simple solution. Here is my attempt to explain it. Perhaps having two different explanations helps.
Imagine you have an oscillator that ticks every 3 units but a clock event that needs to happen every 5 units of time. The key to understanding this is that even though 3 does not go into 5 you can get close without accumulating error.
Consider the first 5 oscillator ticks
  • 3 - less that first time period (5), do nothing
  • 6 - greater that first time period (5), do clock
  • 9 - less that second time period (10), do nothing
  • 12 - greater that second time period (10), do clock
  • 15 - equal to third time period (15), do clock
So at each oscillator tick there is some discrepancy but it does not accumulate. In the example above the clock is 100% accurate every fifteenth oscillation.
With a micro running at many megahertz you can adjust the timer prescale to make this periodic discrepancy as small or large as you like.
In my timer I was happy to be out by a few seconds at any time. Of course you could of course use a smaller prescale value and keep it down to milliseconds if you really need to

Using F# on a Raspberry Pi

This is how I got F# working on a Raspberry Pi. The demo shows how to communicate with a Nokia 5110 LCD (PCD8544). While simple it does show how to use the wiringPi library, GPIO and SPI.

Installing mono

Installing mono on the latest Raspbian is easy. Just run the following Command
   sudo apt-get install mono-complete
That should get everything you need.

Using F#

While there are several tutorials on installing F# on the internet, I've chosen to rather compile the F# code on my main machine and copy the compiled assemblies across. I did this for two reasons
  1. I prefer to work on my main computer, its just faster.
  2. I'm having issues getting the F# compiler to work 100% on the pi, but given point one I've not tried very hard to sort it out.

GPIO from F#

There are several options for accessing the Raspberry PI GPIO ports. I decided to use WiringPi as it seems to be one of the most popular libraries at the moment. There are a few .NET wrappers for WiringPi but I decided to write my own DllImports because its so easy and I can better control how I want the access to work. To get WiringPi working I followed the instructions from http://wiringpi.com/download-and-install/ Once it is installed you will need to create the shared libraries. To do this run
  1. cc -shared wiringPiI2C.o -o libwiringPiI2C.so
  2. cc -shared wiringPiSPI.o -o libwiringPiSPI.so
I'll show you how to use these files shortly.

Nokia 5110 / PCD8544

The Nokia 5110 LCDs are 84x48 pixel LCD screens that are easy to use. There are hundreds of tutorials around so I wont duplicate that effort here. If you want to understand how it works make sure you also look at the datasheet. Wiring will be as follows NB I prefer the GPIO pin numbering over the wiring pi numbering. Remember this when looking at GPIO pin numbers below
LCD Pi
Reset gpio 24 (pin 18)
CE CE0/gpio 8 (pin 10)
DC gpio 23 (pin 16)
DIn MOSI/gpio 10 (pin 19)
CLK SCLK/gpio 11 (pin 23)
Vcc + 3.3v (pin 1)
Light gpio 18 (pin 12)
Gnd ground (pin 6)

Using WiringPi from F#

Using WiringPi from F# is easy enough. Make sure that the two .so (shared libraries) created above are in the same directory as the compiled .exe. Then you can create DllImport statements for the functions that you want to export. As an example here is how to import the "pinMode" function Rather than just using the raw imports, I chose to create a type safe F# wrapper for the imported functions. For example when setting the pin mode I'd like to have an type for the I/O mode so that it can be type checked rather than using an int. Also I want a simple way to specify the pin I'm using that works with the pin number, GPIO number or wiring pi standard numbering. Here is the full code that does all of that A few things to note
  • The DllImport is in a private module.
  • I've created F# wrappers for the imports that use strongly typed parameters
  • I've only defined a few of the pins, enough for this demo

SPI

(See spi.fs)
For the PSI imports I defined a few F# wrapper methods that make working with SPI easier. Remember that SPI is full-duplex so however many bytes you write you will get the same number of bytes returned. However quite often you don't care about the return values, e.g. with the LCD code. So I've created a function that does a write and a read (spiWR) and one that "only writes" (spiW) i.e. ignores the result.

The LCD code

For the LCD I wanted to have the following * Communication using hardware SPI using the wrappers defined above * Reasonably efficient, e.g. send multi-byte commands/data in a singe DC/SPI "session" * A simple interface that hides all of this detail Since you may want to wire the LCD different for different projects the pins used must be configurable. To allow for this there is a lcdConfig record that must be passed to each LCD function containing the required settings. This also documents in code how the LCD should be wired. When you start using the LCD you need to created an instance of this record and call lcdInit. lcdInit will initialise the SPI hardware and send all the required LCD initialisation commands. Remember that to communicate with the LCD you first need to set DC high for data or low for a command. For example lcdGoto lcdGoto, sends two commands (DC is low for both). The first command 0x40+row sets the row. The second 0x80+col sets the column Here is the full code

Using the wrappers

Once all the wrappers are defined the code to interface with the LCD is simple. This demo displays the date and time and updates it every second for ~1 minute,

Compiling the code and copying to the pi

To build and copy the code to the pi I use the following bash script, which assumes the pi is on IP 192.168.0.73
#!/bin/sh
fsharpc --out:lcdDemo.exe --target:exe --debug+ --debug:full --tailcalls+ --optimize font.fs wipiInit.fs gpio.fs spi.fs lcd.fs lcdDemo.fs && scp -r lcdDemo* pi@192.168.0.73:~/prog/fs/wiringPi_5110_Basic/
Make sure you pi a running SSHd. Configuring SSH to allow login using your private key is also a good idea as it means you don't need to type your password each time you build the code. The sample also contains a Visual Studio solution should you be using Visual Studio or MonoDevelop etc. Make sure you copy FSharp.Core.dll and the two .so shared wiringPi libraries discussed above across to the pi in the same directory as the compiled .exe before your first run.

Thats it

Overall getting F# working was pretty simple. The performance is great, startup time is fantastic (the JVM was terrible :( ). I hope this helps get you started. Overall I'm really impressed with how well it all works The full code for this demo can be found here: https://github.com/andrevdm/PiPcd8544Demo

Tuesday 18 November 2014

F# FunScript with NancyFx and Ractive


Getting started


I just started with FunScript and got stuck with a few of the basics. Here is how I got it all working

FunScript is a library that compiles F# to JavaScript. This lets you write strongly typed client side code in F#.  It takes advantage of many of the F# features, async workflows (no callback) etc. Take a look at the FunScript page for more information.

Ractive.JS is a template drive reactive UI library created initially at the Guardian. The Ractive tutorials are very clear and definitely worth using

NancyFx is a lightweight HTTP framework for .NET. It is less popular in the F# world than C# but still works very well from F#.


Read the official docs at http://funscript.info/. The introduction and tutorials are very good and will give you a good background in what FunScript is and how it works.

The code below is based on these demos as well as one of Alfonso's projects https://github.com/alfonsogarciacaro/PinkBubbles.Informa


In this post I'll demonstrate the following
  1. Serving data as JSON with NancyFx
  2. Declaring types in F# and using them in FunScript code
  3. Reading the JSON with FunScript and casting to the strongly typed type
  4. Rendering a basic Ractor template
Although I'm using NancyFx here any web framework should work much the same way.


Creating the solution


  1. Open Visual Studio
  2. Create a new F# console project
  3. Add the following nuget references
    • FunScript
    • funscript.TypeScript.binding.lib
    • nancy.hosting.self
  4. Enable type providers when prompted
  5. Get FunScript.HTML.dll and FunScript.HTML.Ractive.dll.
    I got them from Alfonso's github demo, they are included in my github repo
  6. Download and store the two DLLs in a local folder (e.g. lib)
  7. Add reference to the two DLLs

NancyFx - Serving some data

The code to serve a simple page over nancy fx
Run the projet. If you get permission errors, you will need to run studio as an administrator. Browse to http://localhost:6543/ping This snippet
  1. Creates a nancyFx module "IndexModule"
  2. Defines a route for GET /ping

    This may look a bit strange but all it does is define that whenever a GET request is received for /ping the current DateTime string should be returned. 'box' is called to box the result as on object as that is what NancyFx is returning
  3. Starts the host on port 6543


F# Types


As a simple example all types in the current assembly will be returned. Below is the function that does this as well as the AssemblyTypes record for this data.

Serving the data as JSON



The /data route has been added. It calls the getAssemblyTypes function defined above and returns the data as JSON.
To format the data as JSON you call NancyFx's FormatterExtensions.AsJson method
If you run the project and open http://localhost:6543/data in your browser you should see the JSON data being returned

FunScript - Index page and template


At this point we have a simple HTTP server that can serve JSON data. Now the  scaffolding to get FunScript working needs to be added Create an index.html page that will be used to display the data
This page contains the following items
  • A "ractive-container" div into which the template will be rendered
  • A "ractive-template" script block containing the template
  • A script include for ractive
  • A script include for the generate Javascript from the FunScript compiler. I'll show how this works below

Add the NancyFx route for the index page. I'm loading the page using a File.ReadAllText(). (NancyFx can serve static pages but I'm avoiding the discussion about bootstrappers, resource directories etc here. You should read the NancyFx docs about this if you are using this in a real project)



FunScript - Compiling F# to JavaScript


Here is the F# code that compiles the F# to JavaScript. I'll comment on each section of the code below
Since this is the F# that needs to be compile by the FunScript compiler to JavaScript it needs the ReflectedDefinition attribute on the module
The compile function does the actual compilation. This should be familiar from the FunScript guide
The start function does the following
  •   Creates a Ractive instance
  •   Creates the initial application state
  •   Starts a Ractive loop (mainLoop function)

The application state is stored in record defined like this
Here is the start function. Note the creation of the initial RactiveState. Also note that JavaScript is expecting an array not a list or a seq.

That is the main scaffolding for getting FunScript and Ractive working. Next the mainLoop function which contains the F# code that is going to run in the browser.

This function does the following
  • Check if a reload of the data was requested
  • Call our /data endpoint to get the JSON
  • Cast the JSON data to the stronly typed F# type
  • Mark as updated


This looks a little complex but the core of it is the following
  1. A WebRequest instance is created.
  2. AsyncGetJSON<> is called and casts the JSON to our F# type.
    This is the real magic of FunScript. The AsyncGetJSON makes an AJAX call from the browser for you and then lets you work with your data in a strongly typed way.
  3. The two Globals.console.log calls show the typed data being used.
  4. Finally an updated state is returned


Serving app.js



All that is left is to get the FunScript compile F# served as app.js
Not terribly pretty but simple enough. A text response object is create a containing the javascript compiled from the F# (by the Web.compile function). Then the content type is changed to application/javascript. Finally the response object is returned

If you run the project and browse to http://localhost:6543 you will see your data being displayed. The code itself is mostly scaffolding and is pretty simple. With the basics working you can now continue with the tutorials from Ractive and FunScript to take things further.

Code


The full code is on github https://github.com/andrevdm/FunScriptRactorAndNancyDemo

Feel free to suggest any improvements