Data Collection with ODK


In all projects that involves data collection in large geographical areas, there is always the challenge of

  • Double work since field teams collect data on paper and re-enter from paper to electronic format
  • Data quality is not assured because no constraints etc is enforced on paper data
  • Feedback is not generated quickly since data has to go through long hours of review and analysis usually by SPSS, Eviews etc
  • Relevant data like GPS , video , voice and etc cannot be captured

 

Now thanks to Open Data Kit (ODK) these challenges have been overcomed.

Open Data Kit (ODK) is an open-source suite of tools that helps organizations author, field, and manage mobile data collection solutions. This kit is made up of

  • tools for building forms,
  • mobile application for collecting data using mobile devices ( Phones , Tablets) and
  • a powerful server and data repository, that allows for data visualization on  maps and simple graphs and also allows for export of  data as CSV files for spreadsheets, or as KML files for Google Earth

 

Setting up android mobile devices

 

OdkCollectODK Collect is an android based application that is used to collect the data.  It works on any android device! (Phones, tablets, wrist watches, etc.) , very easy interface for collecting data, it is absolutely free and just like downloading WhatsApp or Facebook onto your phone, you can do same for ODK. Simply search for ODK collect in your Google Play Store.

After downloading, you need to setup the ODK Collect to connect to the server on which your forms are hosted. Read more on configuring the ODK collect application at https://opendatakit.org/use/collect/

 

 

 

 

 

 

 

 

 

 

 

 

 

Designing and building the forms

 

XLSForms

All your paper forms can easily be converted into electronic forms using excel using the XLSForms standard ( http://xlsform.org/). The forms defined in excel will be converted into pure xml.

For testing purposes excek definition of forms converted to xml can be manually added to ODK Collect. Simply place them in the /odk/forms folder on your Android device’s SD card.

To make the forms accessible it should be uploaded to uploaded to the ODK Aggregate

Read more on building forms in ODK XLSForm (https://opendatakit.org/use/xlsform/)

 

Hosting forms and getting insights

 

To make forms accessible online, so that users of ODK Collect can download and use, the forms are hosted online using ODK Aggregate (https://opendatakit.org/use/aggregate/).

ODK Aggregate can be deployed on Google App Engine, which is recommend ,but can also be deployed locally on a Tomcat server backed with a MySQL or PostgreSQL Database either locally or in the cloud

Read more on ODK Aggregate at (https://opendatakit.org/use/aggregate/).

LINGPad , Thank You


 

 

Data Visualised nicely in  LINQPad output
Data Visualised nicely in LINQPad output

Have you been working on larger Dot Net projects and you dare not break any code ? Are you beginner and you want a fast and simple place to run and test your Dot Net codes and see results instantly ?. Meet LINGPad

 

 

Mostly you would want to use some piece of code in you program and you want to know the effect of this code beforehand ? Use LINGPad. In LINGPad you can instantly test a simple code  without declaring Class, Namespace and all those extra codes.

Example Just write the code below in LINGPad and run to see and instant results .

DateTime.Now.ToString("MMMM-yyyy")

The output of your code is also visualised nicely for you.

Thank you LINQPad , because you have made my life easy

Organise Everything with Trello


home-hero

Before

Before I met Trello, I used A4 sheets stippled together for planning, organising, writing down todo items etc. over time, the A4 sheets gets lost, soiled or becomes very disorganised and I mostly cannot make meaning from what I wrote in my own handwriting.  and I loose a great deal of brainstorms, procedure and memory with that.

Now Comes Trello

Trello is the free, flexible, and visual way to organize anything with anyone.
Drop the lengthy email threads, out-of-date spreadsheets, no-longer-so-sticky notes, and clunky software for managing your projects.
Trello lets you see everything about your project in a single glance.

Trello is simple on the surface, but cards have everything you need to get stuff done. Post comments for instant feedback. Upload files from your computer, Google Drive, Dropbox, Box, and OneDrive. Add checklists, labels, due dates, and more. Notifications make sure you always know when important stuff happens.

I am now gladly using Trello to organise my projects. Visit Trello Home and their blogs for more…..

 

How to Send Emails with SMTP in VB.NET


A friend’s office has been using a VB6 script to send Emails for years now. That code needed to be updated and more functionality added. So I took it up and designed a console application to send emails using SMTP in VB.NET.

In this brief tutorial,

  • We shall be using my friends office process as a case study
  • we shall be using Microsoft Visual Studio as our IDE
  • And VB.NET as the programming language.

You can download the project source files here …….. ProjectMol

The case Study

A service is created on the company server to run the project executable. Any time the service runs, the application picks attachment file(s) with a defined file extension, located in a specific folder. The application will then create a mail, attaches the files if they exist, and sends it to a specified email address.

Finally the attachments are copied into an archive folder and the original file is deleted.

Some Features of the Solution

The solution provided in this tutorial has the following features

  • All configurations are read from a configuration file so no need to go into code when configurations changes. By this you can add as many recipients, CC, attachment file paths as you want by just specifying them in the configuration file with a semi-colon separated delimiter.
  • All status messages are written into a log file. By this a folder is created for every day when the application runs and then within that day, all the logs are appended onto a log file of that particular day

Create a Console application project using Microsoft Visual Studio. Launch Visual Studio,  goto File>>New>>Project.  Name the Project as ProjectMol.

Configuration file and Class

We will be putting the entire configuration in an application configuration file. Applications can store their user defined configurations in the appSettings block in the application figuration file.

To do this right click the Project in the Solution explorer and then goto Add >> New Item.  In the General Category of the file templates, select Application Configuration file template. Name it as  App.Config.

This file is basically an XML file that stores the applications general configuration. The App.Config file content is as follows.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
 <startup>
 <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
 </startup>
 <appSettings>
 <add key="smtpServer" value="smtp.gmail.com"/>
 <add key="smtpPort" value="587"/>
 <add key="smtpTimeOut" value="120000"/>
 <add key="sender" value="sender@gmail.com"/>
 <add key="senderPassword" value="MPS123mps"/>
 <add key="recipients" value="recipient@gmail.com.com"/>
 <add key="cc" value="cc@gmail.com"/>
 <add key="subject" value="Sending Mail with SMTP"/>
 <add key="body" value="This mail come from code, thats nice"/>
 <add key="attachmentDir" value="\\192.168.1.2\codec\out\; \\192.168.1.11\coar\out\"/>
 </appSettings>
</configuration>

In order to have access to the configuration in an App.Config file, we will model an Object Oriented representation of the appSettings block. In this class , we shall define a property for each of the configurations in the appSettings block, and a method that will read the XML file and populate the Object’s Properties with their respective data.

Imports System.Configuration
Public Class Config
 Public Property smtpServer As String
 Public Property smtpPort As Int32
 Public Property smtpTimeOut As Int32
 Public Property sender As String
 Public Property senderPassword As String
 Public Property recipients As List(Of String)
 Public Property cc As List(Of String)
 Public Property subject As String
 Public Property body As String
 Public Property attachmentDir As List(Of String)

 Public Shared Function FromConfigFile() As Config
 Dim config = New Config
 Dim settings = ConfigurationManager.AppSettings
 config.smtpServer = settings("smtpServer")
 config.smtpPort = settings("smtpPort")
 config.smtpTimeOut = settings("smtpTimeOut")
 config.sender = settings("sender")
 config.senderPassword = settings("senderPassword")

 config.recipients = New List(Of String)
 config.recipients.AddRange(config.ExtractEmails(settings("recipients")))

 config.cc = New List(Of String)
 config.cc.AddRange(config.ExtractEmails(settings("cc")))

 config.attachmentDir = New List(Of String)
 config.attachmentDir.AddRange(config.ExtractEmails(settings("attachmentDir")))
 Return config
 End Function

 Public Function ExtractEmails(ByVal value As String) As List(Of String)
 Dim v = value.Split(";")
 Return v.ToList()
 End Function

End Class

The CC and recipients configuration are comma separated, this allow for multiple values. The ExtractEmails Method is used to split them in a List of Values

Writing to Log Files

Since the Application is most like to run on remote servers, we need to write down the application logs, so that we can monitor the application via its logs. The code to do so is at follows


Imports System.Net.Mail
Imports System.Security.AccessControl
Imports System.ComponentModel
Imports System.IO

'Imports System.Web.Mail

Module EdiSender

 Sub Main()
 WriteToLog(Environment.NewLine)
 WriteToLog(DateTime.Now.ToString)

 WriteToLog("Starting ProjectMol EdiSender..........")

 End Sub

 Public Function CreateLogStream() As StreamWriter
 Dim d = DateTime.Now.Date.ToString("yyyMMdd")
 Dim info = New IO.DirectoryInfo("./logs/" + d)
 If info.Exists = False Then
 info.Create()
 End If
 Dim file = New IO.FileInfo(info.FullName + "/log.txt")
 If file.Exists = False Then
 file.Create()
 End If
 Dim stream = IO.File.AppendText(file.FullName)
 Return stream
 End Function

 Public Sub WriteToLog(ByVal message As String)
 Dim oldOut = Console.Out
 Using stream = CreateLogStream()
 Console.SetOut(stream)
 Console.WriteLine(message)
 End Using
 Console.SetOut(oldOut)
 End Sub

End Module

So in the CreateLogStream Method, a folder and a log file is created for a particular date. If the file already exists, the existing one is used hence a new one is not created. The WriteToLog simply uses the file returned by CreateLogStream for application logs.

Finding the attachments

Now we will find all the attachment files. If we do not find any attachments, then we will not send the mail. Please note that the file is found based on the file extension .txt


'some codes before this block

Dim attachements As New List(Of String)

For Each item In configs.attachmentDir

Dim attachment = GetFileInDirectory(item.Trim, &quot;.txt&quot;, &quot;n&quot;)

If Not String.IsNullOrEmpty(attachment) Then
attachements.Add(attachment)

End If

Next

For Each item In attachements

WriteToLog(&quot;Attachment File : &quot; + item)

Next

If attachements.Count = 0 Then

WriteToLog(&quot;No attachments were found. Exiting mail sending process&quot;)

Environment.Exit(0)

Return
End If

Finally let’s send the mail

We will then create a MailMessage, add all our recipients, add a subject, a body, and attach our attachments. The attachments files are copied into an archive directory and the version of the file in the original folder is deleted.


Using mail As New MailMessage()

mail.From = New MailAddress(configs.sender)

For Each item In configs.recipients

mail.To.Add(item.Trim)

Next

mail.Subject = configs.subject

mail.Body = configs.body

For Each item In configs.cc

mail.CC.Add(item.Trim)

Next

mail.IsBodyHtml = True

For Each item In attachements

mail.Attachments.Add(New Attachment(item))

Next
smptClient.Send(mail)

WriteToLog(&quot;Message sent&quot;)

End Using

In this tutorial, we have used MailMessage, SmtpClient and DirectoryInfo to achieve our purpose. You can download the whole source files from here, Take time and digest the codes.

Leave comments if you have any question.

PLUG AND PLAY REPORTS TOO [PART 3] : Coding the Plug and Play Engine


K Opoku Asare

In this tutorial, we shall be coding the engine, which will enable us design our RDL/RDLC reports and plug it into our application without having to rebuild the app.

This tutorial is part of a series and so far, we have discussed the following

  1. PLUG AND PLAY REPORTS TOO [PART 1] : The Concepts and Tools
  2. PLUG AND PLAY REPORTS TOO [PART 2] : Designing an RDL report

You can download the full source code here…  PLUG AND PLAY REPORTS TOO SOURCE

STEP 1

With our report opened in Microsoft SQL Server Business Intelligence Development Studio  (BIDS), right click on the report and the select View Code, and we have an  XML schema.

RDL XML SCHEMA RDL XML SCHEMA

Behind RDL/RDLC is XML . To be able to interact with this data structure, we will be modelling it in C# code. By examining the XML, We so many tags but…

View original post 215 more words

Using JSON.Net;


092713_1020_UsingJSONNe1.png

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate.

JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others.

XML is verbose. JSON format is lightweight over XML. JSON is recognized natively by JavaScript. JSON can contain integers, strings, lists, arrays. XML is just elements and nodes that need to be parsed into integers and so on before it can be consumed. XML is still great. JSON’s just the “latest and greatest” compared to XML.

As .NET embraces the world of ever increasing JavaScript based web technologies, frameworks were developed to make .Net work its magic with JSON

[
 {
   "title": "Guides",
   "items": [
               {
                 "name": "getting_started",
                 "title": "Getting Started",
                 "description": "This introduction explains how you can get started."
               },
               {
                 "name": "release_notes",
                 "title": "Release Notes",
                 "description": "This contains release information"
               }
            ]
 }
]

 

As an object oriented programmer, I have chosen JSON and JSON.Net as there is a seamless integration between my data interchange format and my code. I can just serialise and deserialise my Objects and even unanimous objects.

092713_1020_UsingJSONNe2.jpg

JSON.NET has several features:

  • Flexible JSON serializer for converting between .NET objects and JSON
  • LINQ to JSON for manually reading and writing JSON
  • High performance, faster than .NET’s built-in JSON serializers
  • Write indented, easy to read JSON
  • Convert JSON to and from XML
  • Supports .NET 2, .NET 3.5, .NET 4, Silverlight and Windows Phone

Mikhail Tsennykh has done a very good job on using C# and JSON.NET on codeproject

Go through the step he has explained them….

And the world is a better place again.

Javascript Variable Scope : A better Understanding