diff --git a/.DS_Store b/.DS_Store index 1a71cb0..8a26cb4 100644 Binary files a/.DS_Store and b/.DS_Store differ diff --git a/content/posts/find_and_replace_in_neovim.md b/content/posts/find_and_replace_in_neovim.md new file mode 100644 index 0000000..e67bf0f --- /dev/null +++ b/content/posts/find_and_replace_in_neovim.md @@ -0,0 +1,169 @@ +--- +title: 'Neovim Subtitute Magic' +date: 2023-11-15T08:08:49-05:00 +tags: ["snippet", "neovim" ] +author: "Me" +showToc: false +TocOpen: false +draft: false +hidemeta: false +description: 'I was able to speed up some of my workflows by learning how to search and replace specifics in Neovim!' +disableHLJS: true +disableShare: false +disableHLJS: false +hideSummary: false +searchHidden: true +ShowReadingTime: true +ShowBreadCrumbs: true +ShowPostNavLinks: true +ShowWordCount: true +ShowRssButtonInSectionTermList: true +UseHugoToc: true +cover: + image: "nvim_searching_replacing.png" + alt: "Neovim logo with the beginnings of some commands involve search and global" + caption: "Neovim search commands are the best!" + relative: false + hidden: true +--- + +This week, I've had to make some changes to an automation we had setup for a customer in [Workato](https://workato.com). The +original recipe was made by a co-worker with a bunch of javascript nodes. Even though I'm customer facing and generally not +considered a technical employee, I knew the engineer who worked on this was swamped, so I decided to jump in. + +In order to greatly reduce the number of nodes needed in the single recipe, I created a list of dictionaries in python (with +one of the values being another list!) and a quick for loop to grab all the appropriate elements from the various +dictionaries. I had thankfully already copied all the JS nodes to a single file for easy reference and retrieval. That one +file had over 8,000 lines, and I needed to replace all the javascript elements with python. + +Here's one node of Javascript that I was working with: + +```javascript +// @param input fields supplied in the recipe step, as an object +// @return object matching the output schema +// Eg: Code for returning time zone for an IP address + +exports.main = async ({ user_email }) => { + let all_domains = [ + 'domain_one', + 'domain_two', + 'domain_three', + 'domain_four', + 'domain_five', + 'domain_six', +] + + let user_domain = user_email.slice(user_email.indexOf('@')); + let domain_found = all_domains.indexOf(user_domain) > -1; + return { domain_found }; +} +``` + +Most nodes were very similar, all except that their `all_domains` array had anywhere from 1 to 500+ elements. Thanks to the +neovim plugin [Telescope Cmdline](https://github.com/jonarrien/telescope-cmdline.nvim) by [Jon Arrien](https://github.com/jonarrien), + you can very easily see your past history of Neovim commands. I've gone back through and pulled out a few special ones that +helped and that I was very surprised how well they worked. I don't claim that this was the fastest or best way to clean up a +large file with repeating functions, but I definitely learned a bunch about searching-and-replacing in Neovim, and any future +needs for this will be _way_ faster! One thing that has been really helpful with Neovim is that as you run the first portion +of all the commands below, Neovim should highlight what it finds. This is really helpful for real-time debugging of your +regex. + +## Commands and Quick Explanations + +*** + +**Command:** `:g/\//+2/d` + +**Explanation:** The g in the command stands for global (hint, run `:h :g` in vim!) and will search across the entire buffer +you have open. In this command I'm looking for all Javascript single-line comments. Since the forward slash is a special +character for neovim, you need to escape it with a backwards slash, hence the odd looking `\/`. The second forward slash is +the next parameter neovim is looking for, and by adding +{num}, you can search for the pattern plus a number of lines. +Closing out the command (again, next parameter comes after the forward slash) with `d` for delete will then delete everything +with you searched for, plus the additional number of lines. + +Bonus: if you wanted to make sure that you only pulled lines that _began_ with a comment, you'd add a carrot to the beginning +of the command. `:g/^\//+2/d` + +*** + +**Command:** `:%s/let all_domains = /"domains" : /g` + +**Explanation:** Moving on from `g` to `s`, the `s` here stands for substitute. There's one big difference in using `g` and +`s`, though. While `g` will search globally, a singular `s` will only search on your current line. To search the entire +file/buffer, you need to include the percent character before the `s`. The rest of this is fairly straight forward, I am +searching for the declaration of a javascript variable and then substituting it with a python dictionary key. Since both +javascript and python declare arrays with square brackets, I wanted to make sure I kept that after the substitution. Lastly, +unless you only want the subsitute to change the first occurrence of what it finds, you'll need to close out the command with +the `g` flag. Neovim's docs say: + +```markdown +[g] Replace all occurrences in the line. Without this argument, + replacement occurs only for the first occurrence in each line. If the + 'gdefault' option is on, this flag is on by default and the [g] + argument switches it off. +``` + +*** + +**Command:** `:%s/^exports.main.*/{ "uuid" :/g` + +**Explanation:** Really similar to the above command, but in this case, there are no characters I needed to save +post-substitution. We already know what `%s` does, so the pattern I'm looking for is all lines that start with +`exports.main`. Since I don't need anything after that start of the line, I included `.*` which looks for any character until +the end of the line. After the pattern I write out what I need to replace it with, which is the first portion of the +dictionaries I need. Finally, we close it out with a global change so it changes it everywhere. + +**Command:** `:%s/return.*/],` +**Explanation:** Finally, I used this one to close out the domains list/array that I'm using. + +*** + +**Bonus Command:** `%s/"props": \[\(.*\)\]\,/"props" : \1,/g` + +**Explanation:** I won't go over all the characters and regex commands as I'm reusing a lot of them from the above commands. +What is different here is that I've lumped the wild card in parenthesis like so: `(.*\)`. Now, I can call back what is +pulled by using `\1`. The command above is looking for a dictionary item with a list as the value. I didn't need the list, +just a string, since all values in my list of dictionaries are single values. By searching for `"props": [ string ],` and +using the `\1`, I'm able to keep the string intact and the result is `"props": string,`. + +This was the milestone command to learn for me and helped unlock so much potential in my brain for other use +cases. For all the other commands I've gone over above, I'm fairly simply just replacing characters I no longer need that +exist either _before or after_ characters I do need. Now that I can keep strings intact for substitutions, I bet I could +revisit all the above commands and make them even more efficient. + +What I'd like to learn next is if I can use an "or" statement in the regex. Let's say I have the same situation as above - a +list of dictionaries with some dict values being a list. Well, if the lists are all formatted differently, some on new lines, +others on a single line, can I select both in a single command? Something like: `:%s/"domains": \[\(.* OR \n\)/`. + +That will be a post for another time! Let me know on Mastodon if you have any other Neovim search and replace tips and tricks +that have become invaluable to your workflow. I'm always amazed and impressed with this community and how differently people +are able to use the same tool. + +*** + +By the way, here's the final output of running the above commands on the sample code above, converted to Python. + +```python +mappings = { + 'uuid': "1234-1234-1234-1234", + 'domains': [ + 'domain_one', + 'domain_two', + 'domain_three', + 'domain_four', + 'domain_five', + 'domain_six', + ], + 'props': 'property-mapping', +} +``` + +## Resources + +Here are some various links of resources that I found helpful when learning about the above commands. I'm assuming you've +already read through anything in [Neovim's Helpdocs](https://neovim.io/doc/user/) (`:h`), but sometimes you need a different take and explanation. + +* [Vim Search, Find & Replace - Reddit](https://www.reddit.com/r/neovim/comments/i5iptq/vim_search_find_and_replace_a_detailed_guide/) +* [Search Across Multiple Lines - Vim Tips Wiki](https://vim.fandom.com/wiki/Search_across_multiple_lines) +* [Vim Search and Replace with Examples - The Valuable Dev](https://thevaluable.dev/vim-search-find-replace/) +* [Removing Specific Characters in Vim - Stack Overflow](https://stackoverflow.com/questions/62751398/how-to-remove-specific-characters-in-vi-or-vim-editor) diff --git a/content/posts/neovim-a-config-debug-tale.md b/content/posts/neovim-a-config-debug-tale.md new file mode 100644 index 0000000..07a10b4 --- /dev/null +++ b/content/posts/neovim-a-config-debug-tale.md @@ -0,0 +1,29 @@ +--- +title: 'Neovim: A Config Debugging Tale' +date: 2023-10-30T14:03:21-04:00 +tags: [""] +author: "Me" +showToc: true +TocOpen: false +draft: true +hidemeta: false +description: "If you have used Neovim and it's massive plugin ecosystem for any length of time, then you're aware that +debugging just comes with the territory. Here's my tale. Warning: It hasn't been solved at the time of writing." +disableHLJS: true +disableShare: false +disableHLJS: false +hideSummary: false +searchHidden: true +ShowReadingTime: true +ShowBreadCrumbs: true +ShowPostNavLinks: true +ShowWordCount: true +ShowRssButtonInSectionTermList: true +UseHugoToc: true +cover: + image: "neovim-debug-config-tale.png" + alt: "Neovim logo, debugging icon, coding icon, pulse.nvim logo" + caption: "Debugging your Neovim Config... A tale as old as time." + relative: false + hidden: true +--- diff --git a/content/posts/parsing_files_with_python.md b/content/posts/parsing_files_with_python.md new file mode 100644 index 0000000..dba1293 --- /dev/null +++ b/content/posts/parsing_files_with_python.md @@ -0,0 +1,83 @@ +--- +title: 'Using Python to Parse File Contents' +date: 2023-11-02T13:57:07-04:00 +tags: [""] +author: "Me" +showToc: true +TocOpen: false +draft: true +hidemeta: false +description: "I often find myself with various files that need to be parsed and transferred to a CSV. This is how I use +python to parse a long and convoluted file." +disableHLJS: true +disableShare: false +disableHLJS: false +hideSummary: false +searchHidden: true +ShowReadingTime: true +ShowBreadCrumbs: true +ShowPostNavLinks: true +ShowWordCount: true +ShowRssButtonInSectionTermList: true +UseHugoToc: true +cover: + image: "" + alt: "" + caption: "" + relative: false + hidden: true +--- + + +### Full Script + +```python +import csv +import pandas as pd +import re + +LISTTUPLE = [] +LINELIST = [] +COUNT = 0 +DOMAIN_DICT = {} +df = pd.DataFrame() + +with open('./Workflows_js_nodes.js', 'r') as file: + for num, line in enumerate(file, 1): + if "<<<" in line: + LINELIST.append(num) + if ">>>" in line: + LINELIST.append(num) +LINELIST = sorted(LINELIST) +# print(LINELIST) +x = len(LINELIST) + +try: + while COUNT in range(x): + COUNT += 1 + temp_tupe = (LINELIST[0], LINELIST[1]) + LISTTUPLE.append(temp_tupe) + LINELIST = LINELIST[2:] + # LINELIST.pop(1) +except IndexError as e: + pass + +for pagetuple in LISTTUPLE: + res_list = [] + domain_line = int(pagetuple[0]-2) + seg_start = int(pagetuple[0]-1) + seg_end = int(pagetuple[1]-1) + with open('./Workflows_js_nodes.js', 'r') as file: + lines = file.readlines() + title = lines[domain_line][4:-1] + segment = lines[seg_start:seg_end] + for line in segment: + result = re.search(r"(?:'@[a-z|.]+.[a-z]{3})", line) + if result: + res = result.group()[1:] + res_list.append(res) + DOMAIN_DICT[title] = res_list +df = df.from_dict(DOMAIN_DICT, orient='index') +df.to_csv('~/export_file.csv') + +``` diff --git a/static/neovim-debug-config-tale.png b/static/neovim-debug-config-tale.png new file mode 100644 index 0000000..61362d2 Binary files /dev/null and b/static/neovim-debug-config-tale.png differ diff --git a/static/nvim_searching_replacing.png b/static/nvim_searching_replacing.png new file mode 100644 index 0000000..bed3661 Binary files /dev/null and b/static/nvim_searching_replacing.png differ diff --git a/themes/.DS_Store b/themes/.DS_Store new file mode 100644 index 0000000..2356e1d Binary files /dev/null and b/themes/.DS_Store differ