Winclipper 1.1 - A Retrospective

March 23, 2018

I just want to start this post with a massive, “Phew.”

Winclipper 1.1 was an interesting release for me compared to its 1.0 initial release. I built Winclipper for myself rather than for any particular person (the fact that there are any people using it at all is nothing short of mind blowing!), and I was creating it as an opportunity to teach myself C++ and the Win32 API. It took almost exactly one year from me clicking “New Project” in Visual Studio to creating a release tag on GitHub. From 1.0 to 1.1 was a very different journey for me for two reasons:

I knew better how to implement my ideas
I had no idea that I was going to build an update checker

Knowing Now What I Didn’t Know Then

The driving force for why I built Winclipper was work: I have to copy dozens if not hundreds of items in a day for work, and the irritation of losing your clipboard item by accidentally pressing Ctrl-x in Visual Studio due to muscle memory drove me up the wall. Ctrl-z, Ctrl-z, select text, Ctrl-c, find spot again, ctrl-v. I can’t tell you how many times I have had to press those keys, but I have made that error on numerous occasions, and still make the error to this day. I’m not planning on talking about the memory leaks that I fixed: those were easy. This is about about the Win32 limitations that I worked through along the way to building my own clipboard utility. I learned about the shortcomings of Windows native controls, and why Microsoft is moving away from those technologies toward WPF and UWP. The quickest answer to “why?” is that the native Windows controls are very limited. Sure they can be extended to do just about everything, but the support is not native, standard, or even expected by the user. Unfortunately, if you want performance, you have to go low level, and that means that you either create your own controls from scratch (very time-consuming) or you go with the controls that you already have available. The most important control that Winclipper uses is, of course, the menu.

Calculating the incalculable

While I had accounted for what should happen when large sums of clips had been copied, I did not anticipate the strange behavior that the native menu (also known by its window class name of “#32768”, in case you were looking for it) would show when provided more clips than what could be displayed on screen. As you might expect, a pair of little arrows appear—one at the top, one at the bottom—in order to allow the user to scroll the menu. What you might not expect is the inconsistent behavior between the methods GetMenuItemID and GetMenuItemRect: While both return values relative to the top of the menu (regardless of whether or not it is on screen) the latter does not account for the menu having been scrolled, so the RECT that’s returned is for where that menu item used to be. I realized that this was a problem for Winclipper when the preview window appeared next to the wrong clips. First off by one, then two, then ten, and so on. However many lines I had scrolled determined how “off” the preview would be. This happened because I made use of GetMenuItemRect to determine where the corners of the menu item were for the purposes of positioning the preview. So, if I scrolled the menu down by clicking 3 times on the bottom arrow and then hovered on the first visible item in the list (index 3), then the RECT would contain the coordinates of the menu item position that is currently occupied by index 6, and now the preview is three menu items lower on the screen than it should be.

So that’s all terrible, but how did I work around this limitation? BLACK MAGIC 0_0. That is the literal comment that you will find above the following code:

LPRECT hitTestRect = new RECT();
int hitTestDelta = 0;
if (GetMenuItemRect(hWnd, activePopupMenu, 0, hitTestRect))
{
    LPPOINT hitTest = new POINT({ hitTestRect->left + 1, hitTestRect->top + 1 });
    hitTestDelta = MenuItemFromPoint(hWnd, activePopupMenu, *hitTest);
    delete hitTest;
}
delete hitTestRect;

LPRECT menuItemDims = new RECT();

if (GetMenuItemRect(hWnd, activePopupMenu, i - offset - hitTestDelta, menuItemDims))

Note: the if statement leads into the rest of the code that makes use of the menuItemDims variable

Lets break this segment down. First we create a new RECT structure named hitTestRect and declare a variable called hitTestDelta, which we will set to zero. If for some reason we can’t get the the menu item rectangle, everything should still calculate fine (of course, if that code gets skipped, then something weird is going on, but the if is more of just a safety measure anyway). So we attempt to get the menu item rectangle for an item that we know in the active popup menu. Since a menu basically doesn’t exist without 1 item, we get the item in position 0. Now, as you recall, this will be where position 0 was on the screen before scrolling (or we haven’t scrolled yet, and it’s in the same position). We take the top left corner of our hitTestRect and add 1 to the top and left sides. This is the bizarre part: using this coordinate that is slightly within where a menu item was, we perform a hit test using MenuItemFromPoint() to get the position of the menu item that we are actually over top of. After we free up the memory that we were using, we create a new RECT that will receive the position of the rectangle that we really want. Finally, we get the menu item rectangle using the following simple calculation: i - offset - hitTestDelta, where i is the ID of the menu item that was selected (which begins at 1 since an ID of 0 is reserved for the cancel operation), offset is an offset to get the position index of the current menu item (1 if we are on the base menu, or the number of items to show in the base menu if we are in the “More…” menu), and hitTestDelta is the difference that we calculated earlier.

They say, “If it’s stupid, but it works, then it’s not stupid,” but that won’t stop me from calling this entire problem stupid. This was, however, only one of my problems.

Drawing on incorrect assumptions

When I was building out the various features of Winclipper, everything seemed to be working well for the purposes of my tests. I installed it at work (we’re small enough that we still have a mentality consisting of “if it gets the job done faster and it’s free, then use it”) and almost immediately I found that it failed to live up to a few real world tests, namely large amounts of text. At work, we have XML files that are to the tune of 2000 lines long (and that’s a relatively short one!), and mousing over a long clip would activate its preview… several seconds after the clip was hovered over. Unfortunately when I built the preview window initially, I didn’t have a good gauge of how long it takes to copy text from one buffer to another, measure the dimensions that would be required to fit that text into a rectangle, and then reposition/resize both the edit control and the window. Small scale tests don’t always scale up as well as one would like, and that is especially true when trying to calculate the length of a RECT using the DT_CALCRECT option of DrawText, especially when also using the DT_WORDBREAK option to break lines when they reached the rightmost bounds of the rectangle. In fact it turns out that this scales terribly; the operation takes long enough on it’s own, but word wrapping makes the situation substantially worse. Piled on top of that is the time that it takes to copy the text into an edit control’s text buffer.

Before starting work on this problem, I assumed that the issue was with the string copy, so I tried a few things, including some sketchy stuff to reduce the amount of text that I was actually copying (this method would have been vulnerable to buffer overflows and a host of other issues). Ultimately I saw improvement, but not as much as I had been hoping for. Additionally, now my scrollbar that indicated the length of the clip was inaccurate. I then figured that the issue had to be inherent to copying long strings of text, so I assumed (wrongly) that if I could get rid of the copy operation, everything should be fine. To achieve this, I decided to take my usage of DrawText a littler further and paint my own text preview using GDI functions in my handler for the WM_PAINT message. Before I was very deep into this, I could see that it would not solve my issue, but it did improve it significantly. I actually said, “good enough” at this point, and I left it alone until I finished the updater. Upon resuming work on the preview window, I began to suspect that DT_CALCRECT was my bottleneck, so to test my theory, I simply started drawing all of the text to the screen with no rectangles at all. Instantaneous. Then I drew text within the rectangle, but with a fixed size. Immediate. I puzzled over the best way to do this for a few days, but ultimately talking it over with some of my coworkers helped me solve the problem.

When I created the new preview, I added a “+### more lines” label to the bottom to make up for the now non-existent scrollbar. I calculated this number approximately by taking the average height of the capital and lowercase Latin alphabet, and dividing the calculated hight (from DrawText) by that number. When text is not larger than my preset 500x500 limit, it would shrink, but in order to find out how large or small it was, I had to use the dreaded DT_CALCRECT. After doing more testing, I came to the conclusion that I could quickly calculate about 80KB of wide character text before there was any noticeable slowdown. I realized that at that size, text was approaching 2000 lines in length, and knowing the length did not matter much to me for text that long, so I made the decision to get the size of the string (a relatively quick operation by comparison) before even attempting a calculation. If the string was longer than 80KB, then I would skip the calculation entirely and just use the default 500x500 size (which was usually what it would evaluate to anyway). The results were fantastic, and to make up for the loss of the line count in this scenario, I added the total size of the data in kilobytes. I was finally happy with the results.

An updater so simple, even a caveman could use it

While I felt much more confident with my problem-solving skills and more familiar with the C++ language, I couldn’t help but become anxious at the idea that even though I would substantially improve the program with these fixes, at least a few people would probably never see the update if they didn’t follow me on twitter. Naturally the solution to this problem going forward would be to include an updater with Winclipper, but this was met with a problem that quite surprised me. Almost no open source updaters for Windows were easy to implement, and the one that was easy to implement (one I quite liked really)—WinGUp—required a PHP-based site to facilitate update-checking. Since I use Github Pages to host my website and don’t make any profit off my work, I really didn’t want to pay for hosting just for an updater. So because I must be a glutton for punishment (but also for learning 😁), I decided the path of least resistance would be to build my own updater system.

I will say two things from my experience: the first is that building an updater is much easier than one might think, and the second is that building an updater requires much more thought and consideration than one might initially give. There were several updaters from which I drew inspiration. The first was a popular updater on MacOS named Sparkle, from which I took inspiration for the look and feel. The second was the extension system in Safari which makes use of a static .plist file that the browser checks to determine whether or not there are updates available; this was how I knew that such a system was possible. The final updater was WinGUp (mentioned in the previous paragraph) that inspired the function of the updater, that is, simply downloading an .msi file and then launching it to let Windows finish the installation. Fortunately I did not go into this endeavor blindly as I have worked relatively closely with Windows installers at work, and in fact, this entire application did not require very much research in regard to its function. What did require a substantial amount of reading and guesswork, however, was the Windows Presentation Foundation (WPF), which I decided to use to help future proof the design, as well as to teach myself a new skill.

I won’t go into much detail (if I do, that will be in another blog post), but the difficulty I found in learning WPF came down to understanding the Model View ViewModel (MVVM) development pattern, and the limitations of page navigation in WPF (it’s worth noting that UWP has expanded on navigation methods and options, and that I managed to make things work for the purposes of this application). It took me a while to break away from traditional MVC patterns, mostly because I was having trouble visualizing how it was supposed to work at a code level. As a concept, it’s relatively easy: View listens for update events from the ViewModel which does all of the actual work, but this doesn’t give a great idea about how this should be implemented in code. Well, for low complexity application, it might look something like this: A user interface has two fields that allow for integer input, a label, and a button; a ViewModel exists that has three fields and matching properties such as Number1, Number2, DisplayNumber, as well as method named AddNumbers() that adds Number1 and Number2 and stores the result in DisplayNumber (for the sake of this example, you can imagine that a more complex operation could take place with an end result that is some kind of object rather than an integer); the setters of each property fire off a PropertyChangedEvent that the UI subscribes to so that it can update it’s fields; finally the CodeBehind of the View contains the event handler of the button click that makes a call to the AddNumbers() method. The label will get updated automatically once it’s PropertyChangedEvent fires (assuming that it was wired up correctly in our UI designer).

As for the updater itself, like I said, it did not require much research, but it did require consideration. All it does is look to see if a file exists on a server, compare the version in it to the version of the program installed, and then downloads an msi if an update is needed. If you ever decide to build your own updater, you should build it with the understanding that your own websites, services, etc. will likely not exist one day. Also, networks go down in the middle of operations all the time. Also, read/write access violations occur for some people for seemingly no reason. Also developers don’t always check their work for typos (I should know). No matter what, the user does not care about why your updater is throwing an error; they just want their application to work. They should never hear a single peep from the updater until it becomes relevant, even if this means that they never receive another update. Just write errors in a documented log file that a person can look at if they care to. This means that every method that can possibly throw an exception should be handled with a series of try catch statements that write very clear and unique error messages to the log. Admittedly this can make debugging a bit more difficult, but it’s work that must be done if you ever want to use your updater in a production environment.

Don’t be intimidated by large projects, even if they are only for a 0.1 update, just plan carefully and be ready to make adjustments. As for the next update to Winclipper, it will likely consist of a substantial internal restructuring and a handful of easy to implement option additions.