Open Source LabVIEW – How To Contribute

Intro

Open source software projects are making huge contributions around the world. They allow communities to pool their resources and achieve progress that couldn’t be reached by individual teams in silos.

While there are open source projects in LabVIEW, it feels like a resource that we aren’t great at using as a community. Every project I’ve seen seems to handle things quite differently, and it can be hard to track down where to work. Since I started the first draft for this post it seems there have been discussions in the US on the same topic:

In this post (and another on setting up a project). I want to collate what appear to be best practices in other communities to see if we can’t improve the open source capabilities in our community. There are hundreds of guides out there – so I will be liberally linking to others and trying to highlight the potential differences in LabVIEW.

Open Source Project Structure

So what is an open source project?

In short, it is a project that someone has decided to make available for free to everyone to use and improve. I’m not advocating making all of your work free, but often, there are bits of sawdust – the bits of code or tools that you have lying around to make you more effective, that you can share and help drive the community forward.

Firstly let’s look at the people involved in a project:

  • Maintainers – Normally the original developer, they are responsible for the master copy of the project. It may also include trusted contributors who have taken on more responsibility. As well as working on the project, this is the group that will be involved in setting a direction for the project as well as assessing and accepting contributions.
  • Contributors – These are people that have provided code changes or documentation to the project.
  • Users – More obvious, people who are using the project themselves and may contribute bug reports and other issues.

Find Projects

There are some tools for finding open source projects to use or work on, but don’t be surprised if LabVIEW is missing from the list.

If you are looking for a project for a specific function, then Google may be the starting point. However, when I tried this with Modbus, the results weren’t great. Adding “open source” to the search got us closer, we now see download pages, but no source code.

So, to do better, we need to search some common places for open source projects to be hosted. We can stick to Google for this. By far the most common for new projects is github.com so we add site:github.com (bitbucket.org and gitlab.com can be worth trying too). Now we see two open source libraries

Of course, those sites have their search functions which can be useful too.

If you go to Github.com and enter “language:LabVIEW” into the search box, it will show you all of the LabVIEW projects on there. You can then add additional search terms to narrow it down.

A big concern with open source is how do you know if this project is reliable? There are a few indicators that you can consider:

  • Most services have some form of user rating – while the small size of the LabVIEW community doesn’t help these – you may be able to apply some judgement to them. For example, projects can be “starred” on GitHub.
  • Look at how active the project is. GitHub will show you the latest commit, and you can also see releases, commits and contributors at the top. If there is a good commit history or multiple contributors, this is a good sign the project is active which usually means a higher quality output (since maintainers are fixing bugs or improving the code base).
  • Finally, it is open source! Download the code and examples and take a look to see what you think, try it out quickly before committing to it.

Use and Report Issues

The simplest thing you can do to support a project is using it! From the GitHub page you should be able to find directions to install the project and possibly directions on reporting issues.

I try and will be convincing others to do the same, to use the built-in issue tracker on GitHub or bitbucket. This means that:

  • The issues and discussions are public so others can contribute.
  • When you have an issue, you can search this first and see if there is an existing solution. (yay for reduced support burden!)
  • The issues list becomes a list of items that new contributors can work on.

When you find a problem or something you want to change, the first thing to do is create an issue. This allows you to open a dialogue with the project maintainers about the problem. They may already have a fix, or they may suggest you contribute the change yourself.

Fix Issues and Contribute

Firstly, contributions don’t have to be code. I was delighted on the LabVIEW CLI project to have help producing a getting started guide. Maintainers sometimes leave documentation and examples to the end but they are hugely valuable. Of course, you can also contribute to the code base as well.

A good project should have a contributing guide in the repository. This will let you know how the maintainers want to receive contributions. This is also a great place for details on things like LabVIEW version, source code structure or build instructions.

If ther is no contributing guidelines, there are some excellent guides for this already, so I’m not going to repeat them here:

  • A good general description: https://www.hanselman.com/blog/GetInvolvedInOpenSourceTodayHowToContributeAPatchToAGitHubHostedOpenSourceProjectLikeCode52.aspx
  • A LabVIEW specific guide to Github (covers starting from scratch): https://github.com/NISystemsEngineering/GitHub-Hands-on/blob/master/docs/Git%20Basic%20Concepts.md

Further Reading

I hope this article gets you interested in contributing to open source in LabVIEW. If you wanted some more comprehensive guides then I suggest the following:

  • First Timers Only – A guide for those who haven’t contributed to open source before.
  • Open Source Guides – A comprehensive set of guides for open source. The link here is to their contributing section.

But most of all – starting using a project today and let the developers know what you think!

Getting VI Analyzer Into Our Workflow

We have long known that VI Analyzer is a good idea – much like unit testing – people on the other side of adoption swear by it.

We’ve found a few hurdles for mainstream adoption into our process, and I suspect yours too.

1. Understanding Why

The first step in adopting VI Analyzer (and keeping that adoption going) is understanding why you are doing it!

What I mean by this is you need to make the tool fit your process – not the other way round. If you use VI Analyzer because NI says so then you’re going to see less benefit and it will feel a lot more effort.

For me –  I believe that consistent style and code inspection reduces bugs and improves readability of the system. I have a style guideline, but I don’t always follow it. As I typically work on my own, then code reviews aren’t an option.

Your “why” will probably be very similar but the subtle differences will make some of the following items slightly different from mine.

2. Complicated Setup

As I said above, VI Analyzer is all about consistency for us. We want every project to follow the same style guidelines. Unfortunately, VI Analyzer does not make it very natural to create a standard test configuration and share it between projects since there is a single configuration file for test setup and which VIs to test.

These are the steps that I went through to build a standard configuration:

  1. Start with your style guidelines. I made mine into a table and identified what already had a VI Analyzer test, what had a custom test in the community, what could have a custom test and what was not testable.

    Style Guide With Tests
  2. I downloaded the custom tests that are available online and created a couple of key tests myself. I didn’t do them all, and I will expand my coverage over time.
  3. I used a project to create a VI Analyzer configuration file. I worked through the tests and configured them to my style guideline. Then I removed all of the VIs to be tested. I saved this configuration file as my standard.
  4. I created a VI package which would install the custom tests and the configuration file to a shared location in my documents. Full credit to Fabiola De La Cueva and Delacor for this idea. They have been helping their customers with this for some time. (You can see their video of this on Youtube)

By completing these steps up front, I can introduce a consistent VI Analyzer configuration to a new project quickly and easily.

3. Defining A Trigger

As with any good habit, you need a trigger to tell you when to do it else you often won’t follow through. There are a few common triggers that I have seen people using:

  • Post-Commit in a CI Server – I’m not a big fan of this one because you now need another trigger to review the results and implement the changes.
  • Pre-Code Review – This is a great one if you are on a team. You should test with VI Analyzer before a code review. You don’t want to waste time picking up things a machine can identify faster. As I am a solo developer, this one is limited for me.
  • Feature Branch Merge – If you are using a branching workflow in Git or Mercurial, then you can use a feature branch merge as a trigger. This is a good trigger as it should limit the scope of what needs testing. However, if the change list gets long, then there could be a lot to review.
  • Pre-Commit – I use a Javascript IDE that has a tick box in the commit tool to check style before the commit. This workflow is what I want in LabVIEW. Testing at each commit makes the tests quick (since only a few files have changed) and it means that you fix issues while you are in the mindset of that code. It also means that once you commit the code, it is “done”. The main problem with this in VI Analyzer is the execution time of the tests.

4. Speed

I wanted my trigger to be a commit. I feel like the code should go through VI Analyzer to be “finished” and only “finished” code should be committed.

This is a problem though. Analyzing a whole project can take many minutes, and I might commit 10+ times a day.

One way to solve this would be to test only changed code, but VI Analyzer lacks any native support for this workflow.

I have developed a tool to tie it to git changes. It isn’t great, so we haven’t shared it yet (it still lacks many essential features), but it has started us using VI Analyzer regularly.

This tool will take a configuration file and then run it against only the code that Git shows as changed. Testing only the changes cuts the test time and now means that it is possible to check at each commit and fix the changes before that commit.

Ugly – But Basically Works

By overcoming these hurdles, VI Analyzer has become a standard part of our workflow. I hope you can use this to incorporate it as well.

Refactoring: When Should I Create A SubVI?

I don’t write code in the best way the first time. No-one does. Instead, if we want to get to a state of clean code (readable, maintainable etc.), we often have to put specific effort into it.

Martin Fowler’s Refactoring book summarises it nicely when talking about refactoring. You need two hats. One for adding functionality and the other for refactoring. Your process may have a nice point to change hats (I use TDD which does) – or you may have to be more deliberate. But at some point, you need to think about refactoring.

I’m not going to go into loads of detail on refactoring now. Instead, I want to focus on one type of refactoring that I use a lot – creating more subVIs.

Why Refactor Code to have more subVIs?

There are several reasons why code with more, smaller subVIs will tend to be easier to work with than a flat VI:

  1. The diagram is smaller – The simplest of them all – your diagram now fits on the screen!
  2. Improved readability through abstraction – This can be a contentious point. We are putting the detailed implementation another click of the mouse away which some people dislike. My experience is that this is outweighed by how much easier the code is to read. By creating a well-named, cohesive subVI, the calling VI is faster to read since you don’t have to worry about the details. You can worry about the problem it is solving and dig into the details if/when you have to. There are also testing and debugging advantages since the new subVI can be debugged in isolation.
  3. Clear and Obvious Coupling – Coupling is one of the primary concepts in software design that you need to grow to understand. A flat diagram can easily hide coupling in the noise, but once you create subVIs, it is obvious if a subVI has too many inputs or inputs that you wouldn’t expect it to have. These are both signs of coupling problems.

Where to start?

I signed up for a presentation at NI Week to talk about clean code and needed to talk about this topic. The day before planning to write the subVI section, I was working on a customer project. As I have a young baby at home, I try to leave at 5 pm every night now, but it was getting on for 5.40pm, and my Wife was messaging me to see where I was so I got the code running but never put on my refactoring hat.

When I came to write my presentation, I realised that stepping through the refactoring of this code was the perfect example! The code I abandoned that evening is the code in the steps below.

Look for Commented Sections

The first and simplest clue, is to look for sections of code with comments describing what they do. If it is enough of an important chunk to comment, it is probably cohesive enough to make a good subVI.

The Original Code

See the sections for formatting at the bottom? There are three types of data to be formatted for the table. One is already in a subVI but the other two are just labelled as doing the conversion. This is the classic sign of a subVI that hasn’t been created yet.

The first step here is to turn these all into subVIs.

Commented Sections in SubVIs

I’ve created the subVIs, aligned them and given them a consistent naming convention.

Different Levels of Abstraction

I’m still looking for a better term for this, but there is still a smell in the code above for me.

Most of the code is now subVIs doing non-trivial functions. Loading from a log, generating tables. But there is a section on the left that is doing array manipulation. This is a problem because it is too much detail for what this code describes. This code is supposed to “say” load the data from the log, convert the sections to a table and combine them all. But instead this story includes handling memory allocation as well!

So my next step is to abstract this into a more descriptive subVI.

Building The Table In A SubVI

I hope you agree; this code is now much more straightforward to read. It is doing everything it was before (as efficiently as it was) but as developers, we can become faster at understanding the code.

Did you notice the coupling?

This highlighted a nice reason for this refactoring. Why is a build table function using an array of timestamps?

Why Use Timestamps To Build A Table?

We sucked these in as part of the “create subVI” process, but they don’t belong. We can refactor the timestamp array out but using the rows in the time table instead to remove the input.

This is a somewhat trivial example, but by abstracting out the subVIs, we can now understand our code in a way that is clearer than before and better visualise the coupling between functional components.

Are we done?

Naturally there is no “perfect” state. This was the stage I stopped at since I had removed the immediate “smells” and the code worked well. Now the code structure is clearer though it actually highlights some other things we could consider:

  1. The build table code could be generalised for reuse by removing some of the labelling. If I need this same function elsewhere, I may come and grab this code.
  2. The program flow shown is that we generate each sub table before building a large table. It may have a higher performance to directly insert each of the subtables directly into the pre-allocated table. Right now the solution above is simpler, and I don’t have any performance concerns, so I haven’t done this.

I’m going to keep on about this – These potential changes are much more obvious in the new structure than the old – that is why those first steps are important. There is also a point where you have to say good enough, or there isn’t enough information to know what is the next best step (options 1 and 2 above may clash so we don’t do them until we know which we want).

A Note On Testing

Isn’t there a risk of breaking your code with refactoring?  In theory refactoring should not make any functional changes to your code but we have all done things with unintended consequences!

Allowing low-risk refactoring is one of the ways that unit testing leads to better code. Because I have tests around this section of code, I could change it as much as I wanted with confidence that it still works. This is why we treat unit testing as a foundational principle at Wiresmith Technology.

Why Do Your Loops Run?

One of the key architectural rules I learnt in LabVIEW is what I always call in my head the “single source of synchronisation” rule. I need to make two opening statements on this:

  • Someone/Something taught me this – Huge apologies I can’t remember the source of this but thank you whoever you are!
  • Its name makes something simple sound very complicated; it isn’t!

It made an appearance in my post on architectural language so I wanted to expand on it. The quote there was:

A loop within an application component. It has a single reason to run.

Why Loops Run

By “run” what I mean is what triggers an iteration of that loop. I’ve identified these sources:

  • Time – Intended to run every x ms.
  • Event – Either a UI or User Event, the loop contains an event structure.
  • Data – Basically a queued message handler but “data” means anything that means a loop waits for data. A data interface could be queues, notifiers, streams, DAQmx etc. Some external process is forcing us to wait for new data to be available.

The rule means, if one loop is trying to use two of these methods (or multiples of one of these types) then problems may occur. Examples might be:

  • Using the timeout on an event structure (did you know it won’t fire if user events fire, even if you aren’t handling them? Thanks Chris R for this demo!) Thanks Fab for the correction: This was a bug that has been fixed.
  • Using the timeout of a queue to perform a repetitive action.

As you can see – time is usually the conflict because the others are kind of obvious to avoid having in the same loop.

Why This Rule Exists

Put simply, in most cases where two of these exist there is a conflict which is hard to resolve consistently 100% of the time, mostly because the event and data drivers depend on external components which aren’t predictable.

Take the timeout case of the queued message handler. If you want to check a state about every 1 second, then the best case is a new external message once every 2 seconds. Then 1 second after that message arrives you perform the check. Maybe you will get a second check before the next message, maybe not. Most likely you will end up performing the check every 2 seconds. Perhaps that is acceptable?

However then an external component generates messages every 0.1 seconds. Now that check doesn’t happen ever, through no fault of yours! (In the context of the code in this loop).

But I Need To…

Perhaps there is a case where you want to break this rule. I have two solutions:

Proxy Loops

Use a second loop as a “proxy”. For example in the case above have a second loop which runs every 1 seconds and enqueues a message for the check.

Another example is when you have an API which generates messages on an event but you need it in a queue. If you have used the actor framework, this how you extend actor core for GUIs.

Break The Rule… And Be Very Careful

If you don’t see a way to design around this rule, then you need to know you are breaking it and be very careful. Most likely by adding some management code to a loop.

The example I could think of was if you needed data from two loops. In this case, you would write code to read from each queue. If one read timed out, perhaps you store the successful data and then try again, reading only the failed queue.

I don’t think there are many cases, but rules are about knowing when you break them and why!

Whats Your Architectural Language?

The word of the year here at Wiresmith Technology is process. In areas where I have standardised processes life has got easier, less stressful and more reliable. Now I’m looking at the software processes to see where we can get the same benefits.

Something that I have wanted to address for a while is architecture. Working on my own has given me the benefit of being able to be quite ad-hoc and try different designs on different projects.

Well, I often think coming back to your own work after 6 months isn’t so different from working with someone else and I’ve certainly felt the drag of having to review how I built the architecture on each project. So I want to at least have some standard templates.

What I found when I came to it was I first had to define what is in a program!

Language Is Important

There are so many conflicting terms and every framework has its own terminology.

I really wanted to start with knowing what I want in a generic sense. By doing this without looking at specific frameworks it gives me the freedom to find a framework that fits the way I work best (as well as the freedom to change or not use frameworks depending on the project).

I’ve seen this approach work in my business. I’ve been trying to find tools to help me be more productive, but it isn’t until I decide on what the process is that these tools are supporting, I waste hours trying to choose as I have no way to determine what is best!

So before even trying to work with templates or frameworks, I reviewed my previous projects to try and pull out and name the different elements of my architectures so I can map other tools to this.

What I Picked

So here is what I listed as my architectural definitions. Before you read them, understand I am sharing these for your curiosity – you may have your own set of definitions in your team already. This isn’t about right or wrong, this is about consistency between team members and projects.

Term Description Not to be confused with…
Application Component An asynchronous VI with it’s own lifetime and own control of when to run. This is the top level of the architecture design.
Data A piece of engineering data e.g. acquired data. Messages: We split this concept as messages are more framework-oriented.
Message A framework command for a process to do something. Data: although they are data in the strictest sense, they are not directly related to the data involved in the engineering domain.
Message Handler A process that receives heterogenous messages and data. Data-driven process: this has homogeneous data to handle.
Module A set of related code. In our system, it is a class. It is generally unit testable. Module (DQMH), Actor (AF) – These are processes or message handlers.
Process A loop within an application component. It has a single reason to run.

I’m pretty happy with this – the one element of confusion is where the actor style module (whether that is an actor framework actor or another QMH based framework like DQMH). In reality, this sits somewhere between a module and a process but I need to experiment more with how to think about those.

The one I think is particularly important is modules. Too often the important logic gets muddled and mixed with framework code

Next Steps

For me the next step now is to create templates or frameworks to handle these items in a consistent way – more on that in a future post.

My challenge to you is to think about this for yourself. Maybe you already have a framework so you don’t need definitions like this, but where do you find you or your time are inconsistent over time and would a common language help?

2018 Review

Its that time of year – the arbitrary point to review what is going on in our lives! To that end, I thought I would review my LabVIEW life here.

What has worked well this year?

LabVIEW CLI

Probably my most exciting LabVIEW element this year has been working on the LabVIEW CLI toolkit (well it launched in 2016 but let’s not quibble).

It has gained a lot of traction as CI has become more popular in the past 12 months and it is great to have something you have put out there used by other people and even receiving contributions back to the code.

It is so useful that NI are building something similar into LabVIEW 2018!

There are still improvements to be made in handling cases when LabVIEW might already be open, I hope to get to these soon, or we will have to see if NI’s release solves this and so may supercede what we can do.

Getting More Organised

Over the past 12 months, I have better defined how I structure my code and documentation which, while tedious, helps me to focus on the important bits. I’m not done yet – but it is getting better all the time and creates the opportunity for one of my themes for 2018 which will be automation.

What do I need to do better?

System Testing and Analysis

You should hopefully all be aware I love unit testing. It is built into the way I work and has improved my code greatly. I need to take this a step further and make sure I increase my testing at a system/integration level to catch things unit testing can’t. I don’t yet know what that looks like – it is especially hard in LabVIEW since there is so much hardware involved but what I have learnt is I need to try to automate whatever I can to make sure it happens consistently.

Architectural Framework

2018 is the year that I am going to adopt a basic framework at a structural level. This will just save time and increase consistency which will make code easier to maintain and open up more options for collaboration. The only problem is that I’m not a fan of frameworks!

My idea is I need to work out why not. Starting from principles of what I need, I can then evaluate existing frameworks or just build a couple of templates to work from. The best example is that I find most frameworks don’t have the concept of a time-based process, e.g. DAQ, but this is in every system we work on! I’m sure I will share more as I go.

Time for Personal Development

I have found it really hard to date to make sure I spend enough time experimenting, learning and developing tools in LabVIEW (as well as other areas of expertise). This year, I am going to figure this out! Hopefully, that means more posts here as well as this time is the best source of ideas for this site.

What Am I Excited About?

Business Changes

2018 is going to represent something of a new era for Wiresmith Technology. I have decided that I need to focus more on application areas rather than LabVIEW to expand my reach and make marketing much easier.

The area of focus is Automated Measurement Systems. This means an emphasis on applications where we are taking dynamic measurements and analysing or logging this on the fly. While I expect there to be some areas of control in these systems (for example, stimulating the system we are measuring), this represents a move from our previous split of measurements on the one hand, and control on the other.

Why is this exciting from a LabVIEW perspective? This narrower focus also helps me to allocate development time better and prioritise certain technical assets that fit the applications well. It should improve reuse and let me dive deeper into some interesting areas.

VIMs and Channels

My projects all exist in LabVIEW 2015 still, but this year I will move to 2017. While some of these recent releases have been mocked for having very few material changes (I’m looking at you, 2014 icon change!) the few productivity improvements have really made a difference day to day.

2017 also sees some new language features though. I’m excited to try channels – I’m not sure exactly how I will use them yet but I like the idea that the top level of my code can look like an architecture diagram.

VIMs on the other hand look game changing for me. So many re-use ideas are scuppered by the idea that I must create loads of versions for different data types, but this is the solution. Simple things like a “has changed” VI or stall data flow mean that we can produce much more valuable reuse code without the penalty of having to go to variants.

Events!

With a new baby at home, I’m supposed to travel less, but the calendar is already looking at bit exciting! You will find me at:

  • CLA Summit Europe 2018: We are in Madrid this year. I volunteered as co-chair, so I’m going to have to be a bit responsible this year. I’m also going to be banging on about unit testing again.
  • NI Week 2018: I was planning on not going this year due to family and financial commitments, but I got an excellent price on flights yesterday! Skype should keep the family happy.
  • GDevCon 2018: With the lack of a UK event for advanced LabVIEW developers, some of us decided to make our own! Go to the site to sign up for updates on tickets and content.

I hope I will get to meet you at some of these through the year, say hello!

Testing Events In VI Tester

The APIs that you have to test are not always simple. As well as passing data they may involve events (with the front panel or with user events).

The other day I needed to test that an event fired as part of a test case. I could see a generic solution, so I created a template for it. I had two requirements:

  1. If the event doesn’t fire – test fails.
  2. If the event fires with the wrong data – test fails.

In my given when then sequence then we end up with a test that follows the structure:

  • Given: Who knows, in this case, a UI library has been tied to a control.
  • When: We take some action that should cause an event on that control.
  • Then: Check the event.

To check the event we create an event structure outside of a loop as we don’t want to handle multiple events. We need two cases:

  1. A timeout case with a suitable timeout – In this case, we call the Test Case.lvclass:fail.vi to fail the test. This should never run if the when code fired the event.

    Failing Path
  2. A case that handles the event – If you don’t care about the data then you can do nothing here, otherwise, include tests on the data included in the event.

    Passing Path

 

Additional Complexity

  1. Dynamic Event Registration: If this is a user event then you will need to register for the event. I’ve included this in my template, but you must move the event registration to the given case. If you haven’t registered the event before the action in the when case, it won’t ever fire.
  2. Parallel/Dynamic Event Generation: If your event is in some dynamic code you may need to have this running. My advice: DONT. Try and pull out the internal API and test synchronously. Asynchronous testing in LabVIEW introduces timing concerns which make your tests much more complicated.

Where To Get It

If you want to use this template, or even if you are just using VI tester you can download the new version of the VI Tester Advanced Comparisons (VITAC) tool from https://github.com/WiresmithTech/VITAC/releases/tag/v1.1.0.

 

Where Do I Save Config Files In LabVIEW?

When writing applications that will be used by anyone else you will need a configuration file. In my experience, this is almost universal and the more I make configurable, the more powerful the software becomes and the less small changes I have to make for my customers.

Where do we save config files in LabVIEW? The landscape is more complicated than you would think! In this post, I’m going to summarise what we do on our LabVIEW projects. We are focusing on Windows since RT is simpler (put it in /c/) and I don’t use Mac or Linux with LabVIEW.

Types of Config Data

I’m going to refer to two types of config data:

  • Global Data: No matter who logs into the system they should share the same configuration. In my experience, this covers the vast majority of industrial applications.
  • User Data: Configurations that should change depending on the user. This might be screen layouts for example.

Files or Registry?

Microsoft is actually quite keen that you put this data in the registry – that is what it is for. There is a Software folder in each top level folder where you should create your own Company/App folder structure and you can store settings as different variable types.

For user data, you can store it under HKEY_CURRENT_USER and for global data, you can store it under HKEY_LOCAL_MACHINE. In many ways it is a pretty nice solution to the problem, however, I’ve avoided it for 3 reasons:

  1. Files are much easier for users to get, edit or send you. Whilst I don’t want them directly editing the files much it is great that when there is a problem they can send me a file or even a screenshot of the file (when it is readable) so I can understand their setup.
  2. Files make save as… much easier if the user wants to be able to switch between configurations.
  3. Files are universal. Although I don’t have much cross-platform code I like that I can create multi purpose configuration libraries that work on Windows or RT. Without this, I would have to have different code for the different platforms.

I am curious though about who is using this. Please leave a comment below and let me know why you like this and if I have anything wrong.

If Files, Where?

OK, so we have decided on files, where should we put them? Helpfully Microsoft has an article on this however 7 years on there are still issues!

User Data

User data is the easiest and where Microsoft’s advice still works. In each user folder, there is a hidden AppData folder. This is designed to hold user-based configuration files and so the user has full read/write access to this. It is just hidden to protect you from “users with initiative” as Fab puts it in this presentation! Within here you should create a folder structure with Company Name\App Name to follow the standard convention.

To get this path use the Get System Directory.vi with the User Application Data input.

 

Global Data

Global data is where this gets messy. There is an equivalent folder to the user AppData folder for this purpose, but…

In XP all worked well. It was located under All Users\Application Data and all users had write access and software worked.

Then Windows 7 came and two changes occurred:

  1. The location was changed to C:\ProgramData (A hidden folder)
  2. Folders had restricted access. The creator/owner has write access but no-one else.

One use case for this is to install fixed configurations at installation time and this works well since everyone has read access. However, if you need to write these after installation you normally do not have access.

The solution to use this? You need to set the permissions as part of a post install step to allow all users to have write access to the relevant folders.

One day, I may sit down and get this set up automatically as a post install step. For now, I have too many concerns about managing failures of this causing extra support. My solution? Use the public documents folder.

I follow the same structure but in Public Documents instead of Public Application Data. So far I’m happy with this decision and I haven’t had any headaches due to this.

I would love to hear your thoughts. What do you do? Am I wrong?

Implications of WannaCry on NI Based Systems

What do problems like WannaCry mean for us?

The more I learn about cyber security, the more you realise how much it feels like we are on the back foot.

Fundamentally the issue is that the tactics and techniques used by hackers seem to move forward much faster that technology at large with many things we depend on having been designed before security was such a significant consideration.

WannaCry certainly brought this concerns to the forefront again, with legacy systems making the front page. The media scoffed at hospitals using Windows XP still, but in our industry, we know that it is not a simple job to keep complex and custom systems up to date. So what might this mean to the LabVIEW community?

Working with IT More

Antivirus and automatic updates can cause havoc with operational systems but as shown having insecure devices on the network can provide a weak link for exploitation. So while It can be a pain to work with on these systems, we must understand their wider concerns.

We probably need to develop some best practices for system updates – is there a way we can schedule updates to minimise impact? Or can we guarantee the system stays off the network, so it doesn’t risk spreading malicious software? Alternatively, can critical elements be run on LabVIEW RT which will likely require less frequent updates than desktop systems?

Stuxnet showed that you must also consider offline threats, USB sticks will continue to threaten offline systems and if users transfer data to and from systems with them, they must be educated about the risks of using un-vetted USB sticks.

Minimum System Access

I always think one of the best, and basic security practices is that of minimal access. If you don’t need the Web server, disable it. Firewalls should only allow access to required systems, and we have the option to install them to Linux RT targets now.

Critical to this is things like VI server remote access. This allows for arbitrary code execution which is a hackers dream! Make sure you turn it off if you don’t need it. If you do need it, make sure you protect it well.

If you have a multi-device system such as a test rack, then including a router which can provide an internal network with wider access but restrict the external network would be a sensible approach.

 

Minimum access also means only the required permissions for any given user. You should ideally never be running as an administrator as standard. I know it’s easier! But it also makes things much easier for malicious code. When you hit a permissions error, then make sure you give the standard user the permissions it requires. Using Linux trains you well in this and is one of the benefits of learning it. (I know Steve has found it worthwhile)

Examples of where these principles are important are the new Petya variant. The malware spreads through various means. This includes the SMB flaw that WannaCry used, but it will also then sniff the machine for administrator credentials. If it finds them, it will then use these to remotely access other systems that the account has access on, spreading further.

I also have it on my list to look more into the write filters on the Windows Embedded systems which mean that anything written to the disk is only temporary and every reboot brings it back to the original state. The system can still get infected, but it makes a recovery much easier.

Thinking About Recovery

One thing I have learnt over the past couple of years is a backup is only as good as the recovery. If a customer had a machine infected and was losing money while it was down, how fast could you recover it?

I take images of all RT systems, but I am considering whether Windows-based systems should also have an image taken and recovery disk creating on delivery. Then if a machine does get infected (and doesn’t store critical data that has to be recovered first), it can be up and running again in hours instead of days.

I know there are a lot more questions than answers there! But I think it is an interesting discussion to have and something I aim to improve on over time.

By Value vs By Reference In LabVIEW

After my previous post about Learning LabVIEW OOP there were some comments on by reference vs. by value which often come up when talking about OOP. I think there are two reasons that these are tightly linked to conversations about OOP.

  1. In “classical” OOP languages everything is by reference but in LabVIEW OOP is by value. This causes a clash when people have learned OOP from these languages.
  2. We do more by reference work in non-OOP LabVIEW than we sometimes like to admit.

I have been thinking about the techniques and analogs to these lately anyway so this is a bit of a meaty article covering the options that I see for implementing these and some thoughts on how they fit into teaching OOP.

By-Reference vs. By-Value

Lets first define my interpretation of these items. I like to think of them at this level as how data behaves rather than definitions of implementation.

  • By-Value: If you take a wire/data in LabVIEW and change it then it changes only for that piece of code and the code that is dataflow dependent on it. There is also no way another piece of code can change the data on the wire. Branching the wire risks creating a copy.
  • By-Reference: The data of is stored in one memory location. When you make changes they might affect other components that don’t have a dataflow dependency and if you read it twice in a row another piece of code could have changed it.
    (This may differ from a classic computer science definition but my main concern is the behavior I see, perhaps a different term is required but stay with me!)

These strongly overlap with data communication. Another way to think of this is that by-value is communication on a data wire where as by-reference is a tag based communication method (in some situations).

So Why Use By Reference In LabVIEW?

My preference is to always lead with by-value where it works. I think this is key to what makes LabVIEW a powerful language. The data on the wire is yours to do what you want with and you don’t have to worry about side effects when you are programming. I suspect it is one of the keys that makes LabVIEW much easier for people without a software background to pick it up.

There are perfectly valid reasons reasons to use by reference though either in spite of, or because of, these side effects. The following items are cases where I look to by-ref:

  1. Shared Application Resources: This is not a great term but what I mean by this are resources such as an error handler or a system configuration where you want every piece of code singing from the same hymn sheet.
  2. Hardware Resources: This is one of the most common – DAQmx and VISA already all run on a by reference API. If I have to add to the API (attach additional data) I will use a by-ref scheme so it continues to work as you expect.
  3. Huge Data: If a data structure is a big proportion of your application memory you need to make sure you don’t copy it often. You see this in the IMAQ library where images are handled by reference (and the confusion it can cause!). This is the in spite of case – you don’t get any programming benefit but you have to use it due to the constraint of the system.

How To Do By Reference In LabVIEW

There are multiple techniques to get the behaviour I mentioned above. I have put down the key ones below; My favourite will be obvious!

Variables

The simplest and most dangerous technique due to race conditions. This isn’t a problem in single process languages but in LabVIEW it can get you into lots of trouble!

There is one case that I may use them which is for WORM (write once, read many) globals which can be useful for configuration data but I never use them with OOP.

Variables for sharing data

DVRs

Data Value References (DVRs) allow you create your own reference wire to any data type. You access the data through the in-place memory structure which protects the data – no other code can access it until you are finished with it. This is very important for preventing race conditions.

DVRs for by-reference

This is my preferred method for by-ref objects. I would create the class normally but change the standard methods to use DVRs of the class rather than the class itself. What I have found is:

  1. Good – 100% scalable. Want 1/5/20/1000? Not a problem.
  2. Good – The call is synchronous. Once the subVI completes the function it performs is complete.
  3. Good – I have heard criticisms that this can lead to having more wires on the diagram. I think this is a good thing! Wires make it obvious what is coupled to what. Variables and AEs don’t make that obvious.
  4. Good – The property nodes for objects support this with no extra code.
  5. Bad – The boilerplate is tedious! Creating the references and the in place element structures with their weird error handling
  6. Bad – References can be invalid at run time which is avoided with the FGV/AEs.

FGVs/AEs

I’m not going to get into a debate on what term to use for these. What I mean is a non-reentrant VI which uses an uninitialised shift register to store data. Normally developers will have an enum input which defines what function is performed and/or the core FGV/AE is wrapped in another API to allow for an easier connector pane.

These are not traditionally referred to as a by-ref programming method but I would argue they are. The “reference” is which VI you are calling. That is what defines which data is modified or accessed and any changes can be seen anywhere else in the program.

FGV/AE

When I started my company I started down the DVR route already, having experimented with both so I haven’t used this as extensively in big applications but the reason I decided not to start with them were:

  1. No wires so coupling is somewhat hidden (you would have to view the VI hierarchy).
  2. Bloated connector panes (though wrapping it in another API does help with this).
  3. No scalability – you either have to write an addressing scheme into the FGV or maintain multiple versions of the VI.

That said these are the preferred methods of many. It is well understood by different developers and is simpler to create than DVRs and shares the benefits of being synchronous.

It is rare that I use these but one exception is actually when this is the exact behavior I want! If I want a singleton object (for my error handler for example) I create an FGV that stores an DVR/Queue Ref and on the first call it will initialise it. That way I get the same reference everywhere in my application.

Singleton Pattern (based on Aristos Queue Singleton design)

Queued Message Handlers/Actors

This is a bit more of an unusual item to add here but it can be and I believe is used for the same cases as above.

Increasingly these are being used in systems like a module. You have an actor for each instrument you want to talk to for example and you enqueue commands for it to complete.

QMH System (simplified)
QMH System (simplified)

This mode of operation is very similar to a traditional by-ref model. The “reference” is the queue reference or actor reference which ties you back to the “data” stored in the shift registers of the QMH. The QMH loop protects access to the shared resource by processing messages one by one, protecting you from race conditions.

It isn’t exactly like the other options, the key difference is that it can operate independently as well as responding to other requests which can make them hugely more powerful.

There is a major added complexity with this though which is that they are asynchronous. This means two-way communications are difficult for example, what happens if there is an error in the QMH? Also you can’t understand the time from the block diagram. I find I have to create sequence diagrams in order to understand the program flow.

Also you can’t understand the time from the block diagram. I find I have to create sequence diagrams in order to understand the program flow.

This is the reason one of the core tenets of an actor based system is that it is a request and you can’t care when or how it gets done. This rule means you must design your system in order to avoid these complexities, but I think this is a hard rule to follow consistently!

For these reasons, I avoid using these unless I need it to run independently or there is multiple classes interacting within it. I tend to use these for “processes”. For example, a DAQ system where the data just gets published onto an event when it is ready.

So When Do You Teach It?

So coming back to the previous article, when do you teach someone about this?

I would argue that this doesn’t need to be intertwined with OOP. In many cases, if they aren’t new to LabVIEW they will already be using one of these techniques. Sticking with that is the simplest route.

If they are new I believe that the decision between these is probably an architecture decision as each has pros and cons in different scenarios. It is hard to teach them all at once so I would look at your typical architectures. Does one of these tend to form the backbone or default option? (In my case it is the DVRs). In which case I would start with that. You can teach the other methods as they are required. If you suddenly need all four at once you could even hide one method behind another to get a new developer on board quickly.

Let Me Have It!

This is bit of a work in progress in my thought process which the question above prompted. I’m pretty sure my terminology isn’t great but I feel the idea is solid so please comment below or find me on twitter or the NI forums. I find it very helpful to help me understand this better.


By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close