latest posts

While largely quiet on here since my last post two weeks ago I have been hard at work on several smaller projects all of which are on GitHub. As mentioned previously, everything I work on in my freetime will be open sourced under the MIT License.


The first item I should mention is some new functionality in my jcAnalytics library. Earlier this week I had some ideas for reducing collections of arbitrary data down to distinct elements. For example if you had 3 Objects of data, with 2 of them being identical, my reduction extension methods would return 2 instead of 3. This is one the biggest problems I find when analyzing data for aggregation or simply reporting, especially when the original amount of data is several hundreds of thousands or more. I attempted the more straight forward single threaded model, as expected the performance as the number of elements increased was dramatically slower than a parallel approach. Wondering if there were any theories on taking a sampling of data quickly to scale as the number of items increased, I was surprised there was not more research on this subject. Doing a Log(n) sample size seemed to be the "goto" method, but I could not find any evidence to support the claim. This is where I think recording patterns of data and then persisting those patterns could actually achieve this goal. Since every problem is unique and every dataset over time the extension methods could in fact learn something along the lines of "I have a collection of 500,000 Addresses, last 10 times I ran I only found 25,000 unique addresses at an average rate of every 4 records." On subseqent runs, it could adapt per request. Maybe assign Guids or another unique identifier for each run with the result patterns on disk, in a SQL database or in Azure Cache. For those curious, I did update the NuGet package as well with these new extension methods. You can download the compiled NuGet Package here on NuGet or via NuGet Console with PM> Install-Package jcANALYTICS.Lib.


A huge topic in my world at work has been offline/online hybrid mobile applications. The idea that one could "sync" and then pull down data for 100% offline use has been on my mind since it was requested several months ago by one of our clients. Knowing the first approach might not be the best and that I wanted to create a generic portable class library that could be plugged into any mobile application on any platform (iOS, Android, Windows), I figured I would begin my research fully exposed on GitHub and then as stable releases were built I would publish them on NuGet. This project is of a larger nature in that it could quickly blossum into a framework instead of simply a library. As of right now on GitHub I have a the GET, POST and DELETE HTTP verbs working to pull/push data, but not storing the data for offline purposes. I'm still working out the logistics of how I want to achieve everything, but the ultimate goal would be to have any request queued when offline and then when a network connection was made automatically sync data. Handling multiple versions of data is a big question. Hypothetical if you edited a piece of information and then edited it again, should it send the request twice or once? If you were online it would have sent it twice and in some cases you would want the full audit trail (as I do in the large enterprise platform at work). Another question that I have not come up with a great answer for is the source of truth question. If you make an edit, then come online I could see a potential race condition of the data syncing back and a request being made on the same data. Handling the push and pull properly will take some extensive logic and more than likely might be a global option or down to the request type level. I am hoping to have an early alpha of this working perfectly in the coming weeks.


This project came at the request of my wife who wanted a way to view Trendnet cameras from her Nokia Lumia 1020 Windows Phone. Trendnet only offered apps for iOS and Android and there were no free apps available in the Windows Phone marketplace - so I spent an evening and wrote one last August (2014). Again going with the Windows 10 Universal approach, I began to re-write the app to take advantage of all the new XAML and addin features I had long since wanted to add in. Going with my open source initiative, all of the code is checked into GitHub. I am hoping to have everything ported from the old Windows Phone 8.1 app along with all of the new functionality this summer.


Another older project that I see a need to fufill going forward. Since Google Reader faded away, I switched over to feedly, but I really don't like their interface nor how slow it is. Originally this project was going to be an ASP.NET MVC/WebAPI project with a Windows Phone/Windows Store app. As with my other projects, I knew I wanted to simply port over the work I had done to a Windows 10 Universal App, but as I got into working on it, there was no reason to tie the apps back to a WebAPI Service if I did away with the MVC view. Knowing I was going to be freely giving away this application and didn't want to have ads I also didn't want to incur massive Azure fees if this were to take off. So for the time being this project will exist as a Windows 10 Univeral App with full support for multiple devices (i.e. if you read an article on one device, it will mark it as read on the others). You can check out the code on GitHub. I'm hoping for a release in the coming months.


This was a project I had been slowly designing in my head since summer of 2012 - a turn based Star Trek game without microtransactions and the ability for one to simply keep playing as long as they want. I started coding this in August 2014 and into September 2014, but put it on hold to work on Windows IoT among other topics of interest. Now with Windows 10's release on the immediate horizon I figured I should wrap up the game and in kind open source the project. As of now I'm in the process porting over the XAML to Windows 10 as it was originally targeting Windows Phone 8.1. Once that process is complete, I will return to working on the logic and with any luck release it sometime this summer, but in the meantime you can checkout the code on GitHub.


I originally wrote this "game" for my boss's child since there was not a dot math game in the Windows Phone marketplace. Seeing as how it got 0 downloads, I open sourced it. I did start porting it over to a Windows 10 Universal Application, but have not finished yet.


Now that Visual Studio 2015 RC is out, I will more than likely be returning to my open source bbXP project. The only reason I put it on hold was the issues I was running into with NuGet packages in CTP6 of Visual Studio 2015. Coming up in a few weeks is the 20th anniversary of when I wrote my first line of code, expect a retrospective post on that in a few weeks.

A little over a year ago now at work I started diving into my next generation platform. I knew it would have the usual specifications: super fast and efficient, full audit handling etc. The crown jewel was to be a fully automated scheduling engine, but over time the platform itself evolved into the most dynamic platforms I know to exist. Fast forward a few months as the system was in Alpha, I started having my own questions of how to start predicting events based on data points now that I was storing much more data. With several other requirements taking precedence I put it on the back burner knowing the "brute force" method of comparing every value of every object on a per class or report was not ideal nor viable given the amount of reporting possibilities.

Last Sunday afternoon after playing around with Microsoft's Azure Machine Learning platform, I came to the realization that their solution while great for non-programmers I was left feeling as though I could do the same thing, but better in C# and because of that I could integrate it across every device even for offline native Mobile experiences.

Thinking about the question I had a while back at work, how can I predict things based on 1 (or maybe no data points)?

Breaking down the problem - the first thing is to reduce the data set. Given a thousand or millions of rows, chances are there are patterns in  the data. Reducing the individual rows to what more than likely is considerably smaller would make additional processing and reporting much easier and more meaningful. Give a million rows to a C-Level executive and they'll be left wondering what they are even looking, but group the data points into maybe the top 10 most common scenarios and immediately he/she will see value.

At this point this is where I opened Visual Studio 2013 and dove into the problem at hand. An hour later I had a working reusable library: jcANALYTICS. As of right now the .NET 4.5 library does the following:

1. Reduces large datasets into groups of datasets
2. Helper methods to return the most common and least common data rows
3. Solution completer - providing an object, based on the larger dataset, fill in the blanks

As of right it is also multi-threaded (can be turned off if desired). Given 100,000 objects on my AMD FX-8350 (8x4ghz) desktop it processes in  just under 5 seconds - though there is certainly room for optimization improvements.

The next big question I imagine is how will this work with my existing application? Knowing utilization of an external library takes some serious considerations (will the company cease to exist, thereby eliminating support, how much in bed will I be after integrating the framework or library etc.). Well good news - at a minimum to process a dataset just 2 lines of code.

Let's assume you've got some class object like the following:

[Serializable] public class Users {
     public string Username {
     get; set; }
public bool? HasIOS {
     get; set; }
public bool? HasAndroid {
     get; set; }
public bool? HasWinPhone {
     get; set; }
public bool? LivesInMaryland {
     get; set; }
To make it work with jcANALYTICS, just inherit from the jcAnalyticsBaseObject class and mark the properties to analyze with the Tally attribute like so:

[Serializable] public class Users : jcAnalyticsBaseObject {
     public string Username {
     get; set; }
[Tally] public bool? HasIOS {
     get; set; }
[Tally] public bool? HasAndroid {
     get; set; }
[Tally] public bool? HasWinPhone {
     get; set; }
[Tally] public bool? LivesInMaryland {
     get; set; }
That's it, then assuming you've got a List collection of Users you would simply use the following lines of code to process the data:

var engine = new jcAEngine< Users >(); engine.AnalyzeData(users); ]]>
After that you've got 4 helper methods to access the analyzed data:

1. GetMostCommon - Returns the most common data row
2. GetLeastCommon - Returns the least common data row
3. GetGroupItems - Returns the analyzed/reduced data points
4. GetCompleteItem - Given an object T based on the common data, fill in an properties based on the most common data that fits what was passed

I think the top 3 are self explanatory, but here's an example of the last function. Using the Users class above, assume you knew  that they lived in Maryland and had a Windows Phone, but you wanted to know based on the other data whether they had an iOS and or Android device as well like so:

var incompleteItem = new Users {
     LivesInMaryland = true, HasWinPhone = true }
; var completeItem = engine.GetCompleteItem(incompleteItem); ]]>
The engine would proceed to look at all of the data points and use the most probable values.

Going forward, my immediate objectives are to optimize the AnalyzeData method and make the library into a Portable Class Library so I can use it on any platform. Longer term I hope to continue to add more methods to analyze and produce data.

You can download the library from the NuGet Console or from here:

[bash] PM> Install-Package jcANALYTICS.Lib [/bash]