Here is the only frequency that matters:
Do you want to be able to recall the word or not?
Anki almost guarantees that you will be able to remember/recall the word from longer term memory with minimal effort, provided the word is reviewed as the program suggests.
If you don't really care too much about the word, don't bother committing it to a review process and accept that the next time you come across it, there is a good chance you'll have to stop and look it up again. Is that something you don't want to do? Add it to Anki. Is that something you don't really care about? Don't add it to Anki.
Frequency is largely irrelevant as people tend to be interested in reading narrowly rather than widely, meaning the general frequency corpus used in Rikaisama isn't an accurate representation of the corpus of words which are commonly found within the types of things you are reading. A generic "safe bet" is: is the word found in the Core2k/6k/10k deck? If so, you can safely add it to your vocab revising if the concept of generic frequency is of high importance.
So one process could be:
Look up word, check if it is part of the Core2k/6k/10k corpus.
*A. Yes? Add it.
***B1. Add it if you would like to commit the word to longer term memory.
***B2. Ignore it if you don't really care about the word and don't mind looking it up again later.
[Repeat B for each time you come across the word again.]
My process? I like to re-read things at a later point in time and when I come across an unknown word, it is added to Anki if I don't want to look the word up the next time I read that particular manga volume, etc. or not added if I don't care about having to look it up again. That's about as complex as I get, which is kind of like a dumbed down version of the aforementioned process.