Welcome to the final post regarding my Google Summer of Code 2018 project. In this post, I'd like to talk about the overall work product and how it corresponds (or varies) from the original plan. Then, I would like to acknowledge some suggestions of my mentors and talk about some new ideas that were recently discussed with them.
However, before talking about any of those things, I'd like to share the code that was written down in the last twelve weeks. So here is the link to my public repository where all the code can be found and here is a patch that can be merged with the main line of development.
Now, coming to final work product, functionality wise, the feature turned out to be exactly what it was supposed to be, a fast and accurate way to suggest corrections for typographic errors, made while working on the command window of Octave. The difference, however, was in the way of implementation.
My original idea was to make a Neural Network for this problem and I did go to some lengths to make that happen. Precisely, I did collect some data about the most common typographic errors made by Octave users and did code up a small model that could learn the correct spellings of a few commands of Octave. At the time, the motivation behind the Neural Network model was to have an algorithm that could work better than the existing algorithms that are used to compare two strings, in terms of the speed-accuracy trade-off.
However, during the community bonding period, some loopholes in my Neural Network implementation were pointed out by a few members of the Octave community. As a student who wants to pursue a career in data science, those counter points, and further research that was done on the Neural Network approach during the third phase of coding, turned out to be invaluable, for it taught me that 'Neural Networks + Data' is not a magical combination that solves every problem of this world. Maybe they can, but sometimes, simpler, more optimal solutions exist, and in those times, one must look at those solutions and optimize them further according to the problem at hand. Somewhere down the line, it also gave me a better understanding of the nature of Neural Networks.
Now, coming back to the technical details of this project, to summarize it all, I used the faster variation of the edit distance algorithm, the one that uses dynamic programming, and optimized it further by reducing the sampling space on which the algorithm had to work on. To reduce the sample space, I analyzed the data that I had originally collected to make a Neural Network and based on the results of the analysis, I was able to make certain assumptions about the misspellings. These assumptions coupled with some clever data organization techniques helped me code up a fast, and yet very accurate version of the edit distance algorithm. One can read about this implementation in great detail in the previous blog posts.
The plan was to replace this algorithm with Neural Networks, during the third phase, 'if' they happen to perform better. As of now however, I found no way to make a Neural Networks perform better than what had been already made and so the suggestion engine still uses my original algorithm.
Additionally, I had to write the documentation and the tests for all of my code during the third phase of coding and I am glad to say that this work has been successfully completed. The main documentation for the m-scripts can be seen in the help text of those scripts. Besides that, I've also written down the documentation for the database file in a markdown file that is included with the database.
I must acknowledge the fact that Nick had guided me very well on how the documentation should be done, during the second phase evaluations. I did keep his guidance in mind while writing the documentation and the tests, and have, hopefully, made a well documented, well tested product.
Now, although, the main documentation should be enough for anyone who wishes to understand how the feature works, if any additional help is required by anyone, the previous posts of this blog (which contain a very detailed explanation), and the public mailing list of Octave (which I shall continue to follow), should be a good place to visit.
During the community bonding period, Rik and I had discussed the importance of an on/off switch for this feature. This switch was already created by the time the first evaluations took place, but during the third phase, I took some time to wrap up this toggling command into a nice m-script. The users can now do a simple >>command_correction ("off") to switch off the feature and do a simple >>command_correction ("on") to turn it back on.
Next, I'd like to talk about something that Doug recently mentioned to me. He asked me if I could think of some way in which we can track the identifiers that don't get resolved by my software. Essentially, this problem is directly related to the maintenance of the database file. With Octave under constant development, new identifiers will be created and some identifiers will deprecate as well. To make sure that the correction suggestion feature does not loose its value, the database of the identifiers would have to be updated in some regular intervals of time. Maybe an update every 6 months would be enough.
Currently, I've included a markdown file with the database that explains how this update can be done, and for now, this update could be done manually only. For now, I cannot not think of a way in which the database file gets automatically updated. Later on, maybe I or someone else could come up with a way to make a program read the release notices of Octave and its various packages and then modify the database accordingly. Maybe this could be a GSoC project for a future batch of students?
So in conclusion, the planned part of the project is absolutely complete and we have already started thinking of ways in which this feature can be improved. For further testing of the current implementation of the feature, I'd need the support of the members of the community. I would really appreciate it if anyone could try this feature for themselves and see if they could break it, or find any other kind of bugs, or maybe suggest some changes to the suggestion engine that could speed up the feature, or, maybe do something as small as pointing out some pieces of code where the coding style has not been followed properly.
Finally, I'd like to thank the Octave community. Working with them was an invaluable learning experience and I hope to be able to continue to associate myself with them for the years to come. :)
However, before talking about any of those things, I'd like to share the code that was written down in the last twelve weeks. So here is the link to my public repository where all the code can be found and here is a patch that can be merged with the main line of development.
Now, coming to final work product, functionality wise, the feature turned out to be exactly what it was supposed to be, a fast and accurate way to suggest corrections for typographic errors, made while working on the command window of Octave. The difference, however, was in the way of implementation.
My original idea was to make a Neural Network for this problem and I did go to some lengths to make that happen. Precisely, I did collect some data about the most common typographic errors made by Octave users and did code up a small model that could learn the correct spellings of a few commands of Octave. At the time, the motivation behind the Neural Network model was to have an algorithm that could work better than the existing algorithms that are used to compare two strings, in terms of the speed-accuracy trade-off.
However, during the community bonding period, some loopholes in my Neural Network implementation were pointed out by a few members of the Octave community. As a student who wants to pursue a career in data science, those counter points, and further research that was done on the Neural Network approach during the third phase of coding, turned out to be invaluable, for it taught me that 'Neural Networks + Data' is not a magical combination that solves every problem of this world. Maybe they can, but sometimes, simpler, more optimal solutions exist, and in those times, one must look at those solutions and optimize them further according to the problem at hand. Somewhere down the line, it also gave me a better understanding of the nature of Neural Networks.
Now, coming back to the technical details of this project, to summarize it all, I used the faster variation of the edit distance algorithm, the one that uses dynamic programming, and optimized it further by reducing the sampling space on which the algorithm had to work on. To reduce the sample space, I analyzed the data that I had originally collected to make a Neural Network and based on the results of the analysis, I was able to make certain assumptions about the misspellings. These assumptions coupled with some clever data organization techniques helped me code up a fast, and yet very accurate version of the edit distance algorithm. One can read about this implementation in great detail in the previous blog posts.
The plan was to replace this algorithm with Neural Networks, during the third phase, 'if' they happen to perform better. As of now however, I found no way to make a Neural Networks perform better than what had been already made and so the suggestion engine still uses my original algorithm.
Additionally, I had to write the documentation and the tests for all of my code during the third phase of coding and I am glad to say that this work has been successfully completed. The main documentation for the m-scripts can be seen in the help text of those scripts. Besides that, I've also written down the documentation for the database file in a markdown file that is included with the database.
I must acknowledge the fact that Nick had guided me very well on how the documentation should be done, during the second phase evaluations. I did keep his guidance in mind while writing the documentation and the tests, and have, hopefully, made a well documented, well tested product.
Now, although, the main documentation should be enough for anyone who wishes to understand how the feature works, if any additional help is required by anyone, the previous posts of this blog (which contain a very detailed explanation), and the public mailing list of Octave (which I shall continue to follow), should be a good place to visit.
During the community bonding period, Rik and I had discussed the importance of an on/off switch for this feature. This switch was already created by the time the first evaluations took place, but during the third phase, I took some time to wrap up this toggling command into a nice m-script. The users can now do a simple >>command_correction ("off") to switch off the feature and do a simple >>command_correction ("on") to turn it back on.
Next, I'd like to talk about something that Doug recently mentioned to me. He asked me if I could think of some way in which we can track the identifiers that don't get resolved by my software. Essentially, this problem is directly related to the maintenance of the database file. With Octave under constant development, new identifiers will be created and some identifiers will deprecate as well. To make sure that the correction suggestion feature does not loose its value, the database of the identifiers would have to be updated in some regular intervals of time. Maybe an update every 6 months would be enough.
Currently, I've included a markdown file with the database that explains how this update can be done, and for now, this update could be done manually only. For now, I cannot not think of a way in which the database file gets automatically updated. Later on, maybe I or someone else could come up with a way to make a program read the release notices of Octave and its various packages and then modify the database accordingly. Maybe this could be a GSoC project for a future batch of students?
So in conclusion, the planned part of the project is absolutely complete and we have already started thinking of ways in which this feature can be improved. For further testing of the current implementation of the feature, I'd need the support of the members of the community. I would really appreciate it if anyone could try this feature for themselves and see if they could break it, or find any other kind of bugs, or maybe suggest some changes to the suggestion engine that could speed up the feature, or, maybe do something as small as pointing out some pieces of code where the coding style has not been followed properly.
Finally, I'd like to thank the Octave community. Working with them was an invaluable learning experience and I hope to be able to continue to associate myself with them for the years to come. :)
