This week, I’ve had to make some changes to an automation we had setup for a customer in Workato. The +original recipe was made by a co-worker with a bunch of javascript nodes. Even though I’m customer facing and generally not +considered a technical employee, I knew the engineer who worked on this was swamped, so I decided to jump in.

+

In order to greatly reduce the number of nodes needed in the single recipe, I created a list of dictionaries in python (with +one of the values being another list!) and a quick for loop to grab all the appropriate elements from the various +dictionaries. I had thankfully already copied all the JS nodes to a single file for easy reference and retrieval. That one +file had over 8,000 lines, and I needed to replace all the javascript elements with python.

+

Here’s one node of Javascript that I was working with:

+
// @param input fields supplied in the recipe step, as an object
+// @return object matching the output schema
+// Eg: Code for returning time zone for an IP address
+
+exports.main = async ({ user_email }) => {
+  let all_domains = [
+ 'domain_one',
+ 'domain_two',
+ 'domain_three',
+ 'domain_four',
+ 'domain_five',
+ 'domain_six',
+]
+
+  let user_domain = user_email.slice(user_email.indexOf('@'));
+  let domain_found = all_domains.indexOf(user_domain) > -1;
+  return { domain_found };
+}
+

Most nodes were very similar, all except that their all_domains array had anywhere from 1 to 500+ elements. Thanks to the +neovim plugin Telescope Cmdline by Jon Arrien, +you can very easily see your past history of Neovim commands. I’ve gone back through and pulled out a few special ones that +helped and that I was very surprised how well they worked. I don’t claim that this was the fastest or best way to clean up a +large file with repeating functions, but I definitely learned a bunch about searching-and-replacing in Neovim, and any future +needs for this will be way faster! One thing that has been really helpful with Neovim is that as you run the first portion +of all the commands below, Neovim should highlight what it finds. This is really helpful for real-time debugging of your +regex.

+

Commands and Quick Explanations

+
+

Command: :g/\//+2/d

+

Explanation: The g in the command stands for global (hint, run :h :g in vim!) and will search across the entire buffer +you have open. In this command I’m looking for all Javascript single-line comments. Since the forward slash is a special +character for neovim, you need to escape it with a backwards slash, hence the odd looking \/. The second forward slash is +the next parameter neovim is looking for, and by adding +{num}, you can search for the pattern plus a number of lines. +Closing out the command (again, next parameter comes after the forward slash) with d for delete will then delete everything +with you searched for, plus the additional number of lines.

+

Bonus: if you wanted to make sure that you only pulled lines that began with a comment, you’d add a carrot to the beginning +of the command. :g/^\//+2/d

+
+

Command: :%s/let all_domains = /"domains" : /g

+

Explanation: Moving on from g to s, the s here stands for substitute. There’s one big difference in using g and +s, though. While g will search globally, a singular s will only search on your current line. To search the entire +file/buffer, you need to include the percent character before the s. The rest of this is fairly straight forward, I am +searching for the declaration of a javascript variable and then substituting it with a python dictionary key. Since both +javascript and python declare arrays with square brackets, I wanted to make sure I kept that after the substitution. Lastly, +unless you only want the subsitute to change the first occurrence of what it finds, you’ll need to close out the command with +the g flag. Neovim’s docs say:

+
[g] Replace all occurrences in the line.  Without this argument,
+ replacement occurs only for the first occurrence in each line.  If the
+ 'gdefault' option is on, this flag is on by default and the [g]
+ argument switches it off.
+
+

Command: :%s/^exports.main.*/{ "uuid" :/g

+

Explanation: Really similar to the above command, but in this case, there are no characters I needed to save +post-substitution. We already know what %s does, so the pattern I’m looking for is all lines that start with +exports.main. Since I don’t need anything after that start of the line, I included .* which looks for any character until +the end of the line. After the pattern I write out what I need to replace it with, which is the first portion of the +dictionaries I need. Finally, we close it out with a global change so it changes it everywhere.

+

Command: :%s/return.*/], +Explanation: Finally, I used this one to close out the domains list/array that I’m using.

+
+

Bonus Command: %s/"props": \[\(.*\)\]\,/"props" : \1,/g

+

Explanation: I won’t go over all the characters and regex commands as I’m reusing a lot of them from the above commands. +What is different here is that I’ve lumped the wild card in parenthesis like so: (.*\). Now, I can call back what is +pulled by using \1. The command above is looking for a dictionary item with a list as the value. I didn’t need the list, +just a string, since all values in my list of dictionaries are single values. By searching for "props": [ string ], and +using the \1, I’m able to keep the string intact and the result is "props": string,.

+

This was the milestone command to learn for me and helped unlock so much potential in my brain for other use +cases. For all the other commands I’ve gone over above, I’m fairly simply just replacing characters I no longer need that +exist either before or after characters I do need. Now that I can keep strings intact for substitutions, I bet I could +revisit all the above commands and make them even more efficient.

+

What I’d like to learn next is if I can use an “or” statement in the regex. Let’s say I have the same situation as above - a +list of dictionaries with some dict values being a list. Well, if the lists are all formatted differently, some on new lines, +others on a single line, can I select both in a single command? Something like: :%s/"domains": \[\(.* OR \n\)/.

+

That will be a post for another time! Let me know on Mastodon if you have any other Neovim search and replace tips and tricks +that have become invaluable to your workflow. I’m always amazed and impressed with this community and how differently people +are able to use the same tool.

+
+

By the way, here’s the final output of running the above commands on the sample code above, converted to Python.

+
mappings = {
+  'uuid': "1234-1234-1234-1234",
+  'domains': [
+     'domain_one',
+     'domain_two',
+     'domain_three',
+     'domain_four',
+     'domain_five',
+     'domain_six',
+    ],
+  'props': 'property-mapping',
+}
+

Resources

+

Here are some various links of resources that I found helpful when learning about the above commands. I’m assuming you’ve +already read through anything in Neovim’s Helpdocs (:h), but sometimes you need a different take and explanation.

+ + + +