4 Lessons Learnt Unit Testing LabVIEW FPGA Code
After declaring my intent earlier in the year on moving towards an increasingly test driven methodology, one of the first projects I aimed to use it on has been based on FPGA, which makes this less than trivial.
I was determined to find a way to make the Unit Test Framework work for this, I think it has a number of usability flaws and bugs but if they are fixed it could be a great product and I want to make the most of it.
So can it be used for unit testing LabVIEW FPGA code? Yes but there are a few things to be aware of.
1. You Can’t Directly Create A Unit Test
Falling at the first hurdle, creating unit tests on VIs under an FPGA target isn’t an available option on the right-click menu, normally the easiest way to create a test.
Instead you must manually create a unit test under a supported target type but as LabVIEW FPGA VIs are simply VIs, you can then point this at the FPGA code as long as it doesn’t contain any FPGA only structures.
This has it’s frustrations though. You must manually link the VI (obviously) and rename the test. This bit is a pain as I cannot seem to rename new tests without closing and re-opening the project. Re-arranging tests on disk can also be a frustrating task!
2. Test Vectors Are Your Friend
Unlike processor based code, FPGA logic generally works on data point by point and builds an internal history to perform processing. This means you may have to run the VI multiple times to test it well.
The test vectors function of the unit test framework allows you to do this, specifying inputs for several iterations of the VI and being able to test the VI over time.
Mini-tip: Using Excel or a text editor to do large edits can save you losing your hair! (Right-Click and Open In External Editor)
3. Code Resets
Related to 2, because FPGA code commonly uses feedback nodes/feedforward nodes/registers to store data at some point that has to be reset.
The lazy route to this is to simply wire inputs to the feedback nodes however this defaults to a reset on first call mode. This won’t work with test vectors as each iteration is counted as a new call.
At a minimum you should change the feedback node behaviour to initialize on compile or load, but better practice on FPGA is to have a reset input to the VI to allow explicit control of this. Then you must simply set this on the first element of the vector to get the expected behaviour.
4. FPGA Specific Code or Integration Testing
There will be code that the UTF can’t test directly if it can’t run under an Windows target. In this case the best way to include it in automated testing is with a user defined test. This allows you to create a custom VI for the test, which can include a FPGA desktop execution node to allow for testing the VI. Potentially this will have to be accompanied with a custom FPGA VI as well to be able to pipe the correct data into FIFOs or similar.
Hopefully this will help you get over the first few hurdles, I’m sure more will follow and I will try to update them here as I find them.
Richard Thomas
March 16, 2015Great stuff James, as always. On the first pain point, I experienced this in September and it appears to be a bug in UTF: https://decibel.ni.com/content/message/82087#82087
James McNally
March 16, 2015Haha I even remember reading that thread now! I just seem to have managed to find a way to break the workaround.
Fabiola De la Cueva
March 16, 2015James,
Excellent article. Thanks for sharing with us your lessons learned.
I also started a discussion thread at the Unit Testing Group to find out if others are doing Unit Testing for LabVIEW FPGA code and what are they using.
I hope you don’t mind that.
Regards,
Fab
Tony King
December 12, 2016Another pain point I found – normally I want my unit tests to run without a physical target. You can do this by putting the tests under the “My Computer” target; however if VI Server is enabled on the Real-time target then UTF will attempt to run the tests under the Real-time Application Instance – probably yielding issues during testing (I had a lot of linking failures to Real-time only VIs).
The only solution I have found is to disable VI Server (via TCP port) on the Real-time target; then the unit tests run in the right context.
Szymon Ciupak
June 9, 2020Does running FPGA target in simulation mode solve some of these problems?
James McNally
June 9, 2020Hi Szymon,
It may help but I’ve not seen it work yet. It would need the testing framework to understand that first which I don’t think many/any do. I think they would essentially have to be FPGA compatible themselves so the test runner would run in FPGA simulated mode as well which would be difficult.
That said it seems like it opens a technical route to a more realistic test environment if it could be taken advantage of.
However I wouldn’t get too hung up on it. Unit testing is mostly about proving logic and LabVIEW logic largely works the same on FPGA and desktop (AND is still an AND, case structures are the same etc) so even though the execution model is different you can get most of the value from this route.
Szymon Ciupak
June 9, 2020I see, thank you for your insights! At the moment I don’t do much FPGA programming, but maybe one day I will have a chance to check this approach.