Organize Library Member VIs in the Project, Not on Disk

[Darren]

Thanks for chiming in, Joerg. I believe you and I have discussed this topic before.

---

@joerg.hampel wrote:

In my opinion, it makes perfect sense while working in the IDE (i.e. doing the programming) but does not take into account managing resources in SCC.

...

This makes it much easier to understand the nature of resources while outside the IDE.

---

I am a big fan of HSE and the things y'all have accomplished, so obviously y'all have a process that works for you. Please elaborate on what part of your process would be hindered by my recommendations, I'd like to learn more. I have a feeling you and I both regularly work on DQMH-heavy projects, and yet my 'library members are flat on disk' approach has never caused me any problems when working with Git, and I find no real need to "understand the nature of resources while outside the IDE".

[felipefoz]

I totally agree with this, not because I am not a fan of a good organization in disk, but mainly because the IDE doesn't reflect changes in disk, making this a lost battle for the developer.

There are other benefits too, like smaller paths sizes.

I have some questions:

1. Is there already VI Analyzer test for this? That would be nice.

2. I remember seeing this recommendation in the LVOOP Course and it is also part the badge certification for LVOOP, but nowhere else. Do you have a link to a KB or something that list this recommendation?

Regards,

[SteenSchmidt]

---

@felipefoz wrote:

I totally agree with this, not because I am not a fan of a good organization in disk, but mainly because the IDE doesn't reflect changes in disk, making this a lost battle for the developer.

---

If you are a fan of good organization on disk, why would you sacrifice that and follow a recommendation with no organization on disk? Just curious, because I think there is a problem hidden in your statement "making this a lost battle for the developer". What battle is lost? If you're attempting to keep lib and disk organization mapped 1:1, yes, that is futile (and not the goal). But if you organize your disk as you need there, and organize your lib as you need there (and organize your palettes as you need there), then you realize you have 100% freedom to create organization exactly as you require it for your different views. There are different requirements to organization in those different places, but none of them can make do with no organization: The reason we have sub palettes, and the reason Darren is at all talking about organization in the lib (why not a flat hierarchy in libs?), are equivalent reasons that we need organization on disk as well.

Problems with long paths are just side effects. It may force us to give up on disk organization, but it doesn't remove the benefit of disk organization, if there were no issues with long paths. I've heard the same argument for not using OO in LabVIEW: The IDE makes it so cumbersome to create classes and create/change inheritance, that we do not use classes at all. That's a sad argument really, because the benefits from classes (in the right spots) are still there, it just takes a bit of work to reap those benefits.

When maintaining and automating tasks around the physical files (there are dozens of use cases I could mention from refactoring to SCC), I really prefer the first of these two views of a library (and this isn't the biggest library we have, some have +300 files in their debug/dev configuration, this is just 100 files):

Vs:

[felipefoz]

---

@SteenSchmidt  escreveu:

---

@felipefoz wrote:

I totally agree with this, not because I am not a fan of a good organization in disk, but mainly because the IDE doesn't reflect changes in disk, making this a lost battle for the developer.

---

If you are a fan of good organization on disk, why would you sacrifice that and follow a recommendation with no organization on disk? Just curious, because I think there is a problem hidden in your statement "making this a lost battle for the developer". What battle is lost? If you're attempting to keep lib and disk organization mapped 1:1, yes, that is futile (and not the goal). But if you organize your disk as you need there, and organize your lib as you need there (and organize your palettes as you need there), then you realize you have 100% freedom to create organization exactly as you require it for your different views. There are different requirements to organization in those different places, but none of them can make do with no organization:

---

What I meant is that keeping 1:1 is a lost battle, I think we can agree on that.

On the other hand, then I see no point then in doing any extra organization in the disk, besides leaving it flat as Darren proposed.

What you suggest is that there are two types of organization, one in disk, and another in the project, which I hardly disagree, and easily can lead to confusion and besides the one who made the organization, not everyone may understand the logic.

---

@SteenSchmidt  escreveu:

---

When maintaining and automating tasks around the physical files (there are dozens of use cases I could mention from refactoring to SCC), I really prefer the first of these two views of a library (and this isn't the biggest library we have, some have +300 files in their debug/dev configuration, this is just 100 files):

300+ files seems a lot for a library with one scope.

[SteenSchmidt]

---

@felipefoz wrote:

What I meant is that keeping 1:1 is a lost battle, I think we can agree on that.

On the other hand, then I see no point then in doing any extra organization in the disk, besides leaving it flat as Darren proposed.

What you suggest is that there are two types of organization, one in disk, and another in the project, which I hardly disagree, and easily can lead to confusion and besides the one who made the organization, not everyone may understand the logic.

For me it comes down to helping the developer. There are operations that are probably best done with the file explorer as starting point, rather than the library: Moving a coherent chunk of files out of the library (all "GObject" related stuff for instance, well, that's found in the "GObject" folder). It could be SCC tagging or locking. It could be batch modification (callers, icons, descriptions etc.). But also mental mapping of the code for me begins on the disk.

In all honesty, it is rarely a problem for me with keeping a like hierarchy on disk and in the library, if that's the best. But it has often seemed unmanagable when I inherit code that is just in one flat folder. I also see other devs struggle with it, typically experienced by orphaned files in that flat folder which the original dev didn't notice, because they always looked at their code through their library. It might be a ways-of-workings thing, where one way fits your workflow best, and another way fits my workflow best. It could be interesting to get deeper into the "why" of these habits 🙂

---

300+ files seems a lot for a library with one scope.

---

Oh, totally. 300 is an outlier. I just went through a couple of typical libraries here: 6, 36, 46, 58, 88, and 94 files. And I wrote "in their debug/dev configuration". Our libraries have all source code in them, including unit tests and utilities only meant for the IDE (help menu, tools menu, probes, quickdrop and so on). Our packages are built with several different configurations in one go, from the same source. For instance:

- A Debug build of the package contains all those files, and is meant for use in the IDE. It has block diagrams, unit tests, palettes, and all that.

- A Developer build of the package is stripped of test code and does not have debugging enabled. We switch to Developer builds when we're ready to build a runtime application.

Furthermore, our packages contain two libraries (or will, when next release of our package template is rolled out): An "Application.lvlib" which contain all the code that may end up in an application, and a "Utility.lvlib" that contains all the stuff that only makes sense in the IDE. Only stuff from "Application.lvlib" is available through the palettes and quickdrop. This way AppBuilder does not drag anything from "Utility.lvlib" into the application. So a package with 100 source files will probably end up only dragging 30 files into your application, once test code* and "Utility.lvlib" have been (automatically) removed.

* Test code is kept in "Application.lvlib" and not in "Utility.lvlib", because then it's possible to build an application instrumented for debugging just by having Debug packages installed when building your application.

[Taggart]

---

@SteenSchmidt wrote:

---

@felipefoz wrote:

I totally agree with this, not because I am not a fan of a good organization in disk, but mainly because the IDE doesn't reflect changes in disk, making this a lost battle for the developer.

---

If you are a fan of good organization on disk, why would you sacrifice that and follow a recommendation with no organization on disk? Just curious, because I think there is a problem hidden in your statement "making this a lost battle for the developer". What battle is lost? If you're attempting to keep lib and disk organization mapped 1:1, yes, that is futile (and not the goal). But if you organize your disk as you need there, and organize your lib as you need there (and organize your palettes as you need there), then you realize you have 100% freedom to create organization exactly as you require it for your different views. There are different requirements to organization in those different places, but none of them can make do with no organization: The reason we have sub palettes, and the reason Darren is at all talking about organization in the lib (why not a flat hierarchy in libs?), are equivalent reasons that we need organization on disk as well.

Problems with long paths are just side effects. It may force us to give up on disk organization, but it doesn't remove the benefit of disk organization, if there were no issues with long paths. I've heard the same argument for not using OO in LabVIEW: The IDE makes it so cumbersome to create classes and create/change inheritance, that we do not use classes at all. That's a sad argument really, because the benefits from classes (in the right spots) are still there, it just takes a bit of work to reap those benefits.

When maintaining and automating tasks around the physical files (there are dozens of use cases I could mention from refactoring to SCC), I really prefer the first of these two views of a library (and this isn't the biggest library we have, some have +300 files in their debug/dev configuration, this is just 100 files):

Vs:

---

You don't have one library there. You have 13 all stuck into 1.

I've heard of God classes before, but never God libraries...

[Taggart]

---

@joerg.hampel wrote:

For the record, I don't agree with this recommendation, either. In my opinion, it makes perfect sense while working in the IDE (i.e. doing the programming) but does not take into account managing resources in SCC.

At HSE we have a rule of storing all manually created VIs for a DQMH module in a subfolder on disk, independently of where they are stored inside the library. This makes it much easier to understand the nature of resources while outside the IDE.

---

I'm curious why you need to "understand the nature of resources while outside the IDE"

Can you elaborate?

[SteenSchmidt]

---

@Taggart wrote:

You don't have one library there. You have 13 all stuck into 1.

---

So you would in all seriousness create a library and a VI package for 1 and 2 VIs?

Scripting is a special case for a library, that I showed here to illustrate what a flat structure would do to loss of overview. It might get split into more libraries when done, but the example stands: You had no idea which of the "13 libraries" each of the shown VIs belonged to, right?

[Taggart]

---

@SteenSchmidt wrote: So you would in all seriousness create a library and a VI package for 1 and 2 VIs?

I must have missed the memo on minimum size for libraries.What is so hard about creating an .lvlib? It's like 1 click in the editor.

I have certainly made libraries with as few as 1 or 2 VIs. Sure. I even released one. My approval tests library has exactly 2 VIs. It's on VIPM.

If you go back to Steve's coupling and cohesion, if you have to separate a library into 13 different folders on disk to organize it, that should be a sign that it is not very cohesive.

[SteenSchmidt]

---

@Taggart wrote:

---

@SteenSchmidt wrote: So you would in all seriousness create a library and a VI package for 1 and 2 VIs?

I must have missed the memo on minimum size for libraries.What is so hard about creating an .lvlib? It's like 1 click in the editor.

I have certainly made libraries with as few as 1 or 2 VIs. Sure. I even released one. My approval tests library has exactly 2 VIs. It's on VIPM.

If you go back to Steve's coupling and cohesion, if you have to separate a library into 13 different folders on disk to organize it, that should be a sign that it is not very cohesive.

---

A (code smell) sign is not the same as an absolute - just as there isn't a "memo on minimum size for libraries". It would definetely not be the goal to have as many and as small as possible libraries, with hundreds of possible dependencies as result, right? There are recommendations that work most of the time, and there are exceptions to such best practices.

I have said two things here, that you seem to ignore to pick a fight: 1) I chose the library here as an example to illustrate the difference between folders and no folders, nothing else. You may think that this is rubbish code, fine, but similar examples are abundant out there: DQMH, Caraya (400+ files in the main lib), NIAF... almost anything that you look at exhibit the same issues even if we're just talking 40 files. And having libraries in libraries in libraries are just hiding the fact they tried to segregate but ended up compiling anyway, or folders named "subVIs", "controls", or "classes" just add no information. 2) My example is a work in progress, it may very well get refactored into multiple libraries once done. Funnily enough, that final refactor is made so much easier because the files are in folders on disk.

We could've had many fruitful exchanges to understand and learn from each other, but Sam, we've been here before, I won't go into a fight with you again where you cherry pick stuff to misunderstand on purpose for some reason. It's clearly personal, so I'll stop from communicating with you in these forums from now on.