The Complete Guide to Keyword Research For SEO

All content campaigns begin with the same thing:

keyword research.

And there’s a reason for this. Targeting keywords will lead to long-term and, possibly, short-term organic search traffic for your website.

The bad news is that 80% of SEOs and marketers do keyword research wrong.

They plug in a main keyword into Google’s Keyword Planner, download the results, and then start sorting through them in a spreadsheet.

But here’s the thing: literally thousands of other marketers and SEOs have already searched that exact same keyword.

Unless you’re very lucky, you won’t find anything resembling a low-competition keyword to target.

Now, if you have a site with the authority of Quick Sprout, that’s not a big deal. But if you’re working with a less authoritative domain, or a brand new one, you need to find realistic keywords to rank for.

The good news is that the keyword planner only shows a very small portion of keywords you can target.

If you can find “hidden” keywords that have a decent search volume, but less competition, your organic search traffic will grow rapidly.

We’re going to walk you through 5 modern keyword research methods that you can use to uncover keywords that haven’t been targeted by hundreds or thousands of your competitors.

You’ve all used Google AdWords keyword tool before, right? We’re not going to cover that here though. We’re going to go beyond the common keyword research tools and explore some less talked about ways to find the best keywords to target for your site.

Understand what keyword research really is

Keyword research is an art, not a science.

While there are some good tools out there, they should be the start of your research, not the end.

Most SEOs use keyword research to figure out how much search traffic they could potentially attract for free.

Experienced SEOs use keyword research to understand their readers.

Keywords, and their search volumes, tell you a few things:

1. What your audience’s main problems are – if 5,000 people are searching for a particular keyword, it shows that it’s a common and important problem in your niche. Create your content around the most popular problems.
2. The language they use – people describe problems in different ways. If you can understand which words your reader uses to describe problems, you can write more compelling content.
3. Which topics are becoming more important or less important over time – as niches evolve, keywords gain and lose popularity. Good keyword research will help you identify which searches are becoming more common so that you can create more content around those.

The methods in this post aren’t meant to help you find basic keyword phrases like “best toaster.” They are meant to help you understand what’s important to your reader and help you create a reader persona.

Once you understand how your reader searches for help, you’ll be able to find untapped keywords that are perfect to target.

Students are your best audience

You do online marketing to sell something.

Most often, it’s knowledge.

Not everyone will pay for knowledge; many are happy to try to soak up everything for free.

But your most valuable readers are those that know that paying for high quality training is worth it. They get the highest chance of success and will save a lot of time as well.

This ideal type of reader mainly hangs out in a few different places:

• Large ecommerce stores – they buy books and audiobooks from marketplaces like Amazon
• Blogs – they follow influencers in a niche and buy training courses directly from them
• Education marketplaces – the demand for high quality online training is rising fast. Sites like Udemy and Skillshare sell thousands of courses to eager students.

With this method, we’re looking at the third location listed above. Users of Udemy are highly motivated learners and are ready to pay significant amounts for good training.

Even though you’re not selling anything yet, you want to create content around keywords that will attract this audience.

That way, when you are ready to sell, you have a highly receptive audience.

There are two main ways that you can do keyword research on Udemy.

Option #1 – Look at what students already like

The top courses in each category on Udemy have a few important things in common.

First, they are created by experts in a niche. These are the people who have been writing and teaching for years, so they know the language and phrases that the audience typically uses.

Secondly, we can sort courses by popularity. By looking at the most popular courses, we can see all of the most taken courses.

What this means for our keyword research is that the course participants relate strongly to the copy for the course. We can generate keyword ideas from these sales pages.

Let’s go through this process step by step.

First, click on “browse courses” after you have created an account or logged in (free).

This will bring up a menu on the left side with all the different categories of courses:

In this post, we’ll use a marketing blog as an example site to conduct keyword research. So, from the list above, we would choose the “Marketing” category.

Once you pick a main category, you’ll get a list of featured courses. You can choose to drill down a bit more. We could pick digital marketing, SEO, or social media marketing if our blog had a more specific audience:

Whether or not you pick a sub-category, the courses you see aren’t necessarily the most popular courses, so we don’t want to look at them.

Instead, pick “All Courses” from the top menu, and change the “sort by” option to “popularity” on the next page:

You’ll notice that you have additional filtering options on the side.

The most interesting option to me is “level,” which describes the difficulty of a course. If we created content for beginner marketers on my blog, we would want to look at the most popular courses for beginners.

But if we wanted to create content for experts, we’d want to filter the results down to “expert” level courses. As audiences get more experienced and advanced, they typically use different keywords to describe their problems.

It’s important because you don’t want to create content that solves a problem that experts have using the language a beginner would use. Your content won’t attract any expert marketers, and even if it did, they wouldn’t be impressed by your writing (i.e., it would not resonate with them).

Once you’ve filtered down the courses to only the top courses your audience would be interested in, we can finally find some keywords.

Here’s the start of the course description for the first top course we chose:

You’ll notice that the author already bolded most of the keyword phrases important to this audience. He knows the phrases potential students are looking for (from years of experience) and makes them stand out.

Not all course authors will do this, but this is an area where you can find several keywords.

At this point, you want to start recording your keywords. You may directly target them later, or you may use them to generate additional keyword ideas to target. For now, just start a spreadsheet with them.

Note that you don’t have to copy them down exactly as they appear. For example, we doubt that anyone is searching for “how to attract ultra targeted traffic.” But a phrase such as “how to attract targeted traffic” is likely a fairly common one.

Once you’ve finished extracting keywords from the course description, scroll down to the “curriculum.” It has a list of each “lecture” in the course.

These will typically be on the most important (and alluring) topics for potential customers:

From these first three sections, we would add the following keywords to our spreadsheet of keywords:

• What is list building
• Why you must build a list
• Tools to build a (email) list
• How to create a sales funnel

Again, take out any word(s) that aren’t going to be used very often by searchers.

What we just showed you was part of the sales page for a single course. You can often generate dozens of quality keyword ideas for your list from a single course. Go through as many courses as you like until you start seeing too many duplicate keyword ideas.

Option #2 – Look at what students are searching for

Any large website that sells something typically has autocomplete suggestions because having this feature usually increases conversions.

Udemy is one of those sites that provide suggestions in its search bar.

When you start typing a word, it will show you the most searched terms containing what you’ve typed so far:

We would write down all of these keyword suggestions on my spreadsheet for later. These are very useful because they are already ranked from most searched to least searched.

Finally, don’t just stop with your main keyword. If you start typing in any of those suggestions, you may be able to get more suggestions.

For example, as we started typing “marketing free,” a suggestion for “marketing freelance” came up, which would make a good addition to our list.

Find the questions your readers are asking

The best keywords are the ones that come from your readers themselves.

Years ago, it was difficult to find them, but now you have quite a few options at your disposal.

Here are three different ways to find questions that your readers are asking so that you can extract additional keywords.

Option 1 – Faqfox

Faqfox is a handy little tool that can be used to generate content ideas or keyword ideas.

Based on a keyword you enter, the tool will scrape a list of threads on various forums and aggregators.

To start with, enter a keyword in the first text box. Then, you can either enter a forum (which may or may not work) or choose one of the pre-made categories of starter sites.

Once you click on one of the categories, URLs of relevant forums and subreddits will be loaded into the tool automatically. If you can’t find an appropriate category, choose “generic,” which will search all the biggest sites such as Reddit and Yahoo Answers.

Once you click the search button, you’ll get a long list of thread titles (with links) relevant to your keyword.

From this list, you can start picking out keyword phrases and full questions that your audience has already asked in other places:

You get a list of at least 100 threads for your keyword. Just from the results in the picture above, we would write down:

• Which email marketing service should we use
• My Gmail email is going to junk folders
• Video marketing platform
• How to get a job in advertising
• How to take over a webmastering job
• Free online marketing tools

All of these are potential keywords to target in future content.

Option 2 – Scrape a forum

A tool like Faqfox covers all the biggest forums and Q&A sites—but not all of them.

If you find a forum in your niche, you can look at the threads just like we did above to extract keywords for your list.

In addition, you can learn more about the kind of language your readers use.

To do so, we’re going to copy thread titles and then feed them into a keyword density tool. By looking at the words and short phrases that are used most often, you will begin to see a picture of what matters most to your audience.

For example, let’s say you wanted to learn more about the audience in the main Internet marketing section on Warrior Forum.

First, highlight all the thread titles:

Copy and paste this into a blank spreadsheet, and then delete all the junk data:

Copy the remaining data (first column), which should be all of the thread titles.

Paste them into a free keyword density analyzer tool—just make sure you select the “text” option:

All these tools work the same, so use any you like.

Once you run the tool, you’ll get a long list of single words, 2-word phrases, and 3-word phrases listed in order from most used to least used:

To be honest, our results aren’t great in the screenshot above. That’s because we only scraped one page of forum results. Ideally, you want at least a few hundred so that trends start to become more obvious.

Still, you can learn a little bit about your audience from a brief analysis like this. For example, they likely use the word “best” instead of “top.” In addition, they are interested in “finding” answers to their problems.

Option 3 – Q&A sites: The last method we’ll describe here to find questions from your readers is by going on Q&A sites such as Quora.

Quora is probably the biggest, but there are other question and answer sites you could use for research:

• Yahoo Answers
• Stack Exchange
• Wiki Answers
• Ask Metafilter

The reason that Q&A sites are often more useful than forums for keyword research is that unlike forums, which are ordered by the last reply, Q&A sites have algorithms to show you popular content, both past and present.

The first step is to enter your main niche into the main search bar.

Use the filter on the left to narrow down the results to “Topics.” Topics are basically collections of all the questions in a particular niche.

You can follow the topic, which will make it easier to monitor new questions in the future. Or you can just click the link to the main topic for now.

When we clicked “Marketing”, which is a huge topic (1.7 million followers), it gave me several smaller categories to choose from.

Right from these small category previews, you can already find a few good keywords:

Once you’re done mining that page for ideas, click one of the sub-topics to dig in a little further.

In our case, we clicked the main “Digital Marketing” header:

As you scroll down, you’ll see a mixture of new and old questions.

You can look at the number of upvotes each question has (bottom left corner of any question) to see if it’s popular. Ideally, look for questions with at least a few upvotes.

Go through as many questions as you like to find keywords. You can easily find 50-100 to add to your list with this method.

Steal your competitor’s accidental keywords

Large sites, the leading sites in your niche that produce a lot of content, target a lot of keywords without doing it on purpose.

They may mention one in a headline of an article and/or a few times throughout the article.

You can’t compete with these sites for the main keywords they target. However, if they rank for any of these other keywords, you can try to overtake them.

If you create great content around these keywords, you will have better content for those particular keywords because it is more targeted.

This is a difficult method to use, but it can provide some useful results. It’s best used as a second or third option to the other methods in this article.

Step 1 – Use Google’s Keyword Planner

Go to the keyword planner and paste the URL of a competitor in the space where it asks for “your landing page”. Leave the keyword textbox blank. The main blog URL typically works best.

Google will return up to 800 keywords that it associates with the site, sorted by relevance.

You can also input a specific URL of a blog post into the landing page area. For example, if our new hypothetical blog was going to focus on social media marketing, we could enter posts about social media from Quick Sprout into the tool:

You will get some keyword ideas that wouldn’t have come up if you just typed “marketing” (or related) into the keyword box of the tool.

We recommend downloading all the results into a spreadsheet (“Download” button in the top right corner). Copy any keywords that seem good into your main list.

Step 2 – Plug these keywords into a rank tracking tool

One of the main ideas behind this strategy was to find keywords that are easy to rank for. If a competitor can rank for a keyword without even trying, it should be easier for you to rank highly for it.

Just because the tool associates those keywords with the site doesn’t mean the site ranks highly for them. You’ll need to plug them into a rank tracking tool or check them in Google manually (time-consuming).

For the sake of this example, we used Pro Rank Tracker. The free plan will give you 20 keyword results, but you’ll obviously need a premium account if you’re taking this seriously.

Add in your competitor’s domain along with the keywords from the previous step:

Click the “Save” button, and give the tool a minute to check the ranking for those keywords.

When you go to the URL view in the side menu, you’ll see all the keywords you just entered under your competitor’s URL:

Ideally, you’re looking for keywords that are ranking in the first two pages and aren’t fully included in the title. Those are typically the best candidates.

Don’t put all your eggs in one basket

All keyword research methods and tools have their own limitations.

In order to get the most complete set of keywords, you need to use at least a few different tools.

Now that you have a large list of potential keywords to target, you should run them through a keyword tool to get even more results that might not show up on a standard Google keyword planner list.

There are a few different tools you can use for this.

Tool #1 – Keywordtool.io

We told you about how powerful the autocomplete suggestion feature on large sites can be—none more so than Google itself.

This keyword tool types in your keyword plus different letters, one at a time, and records the autocomplete suggestions.

Type in any of your keywords (from your spreadsheet) into the tool, and click the search icon.

This will bring up a large list of keyword suggestions:

Not all of them will be good, but you’ll be able to find quite a few keywords to target.

There are other tools that utilize this same method, e.g., Ubersuggest:

Tool #2 – Term Explorer

This tool seems to use data from Google’s keyword planner but is able to give you more than the standard results for a keyword.

Once you create an account, select the bulk keyword tool either from the dashboard or the top menu:

Give the job a name, and then enter one or more of your keywords.

Term explorer is a really powerful tool if you have a paid account. Free accounts are limited to tiny jobs (1,000 keywords), which is still better than a basic Keyword Planner search.

However, paid accounts can change the results size to up to 90,000 keywords, which will reveal all kinds of hidden keyword gems.

After you run a job, a list of keywords will be generated, similar to the Adwords display.

You can filter the results with the left sidebar if needed:

If you start with a broad term (such as “marketing” in my example above), don’t stop there.

Pick some of the best long-tail keywords, and create another job for them to reveal even more unique ideas.

We created a new one for the keyword “internet marketing strategy” and ended up with a few long-tail keywords that would probably be good to target.

Keyword research and competition go hand in hand

Keyword research will allow you to compile a list of hundreds or thousands of relevant keywords.

However, this is useless until you assess the competition level of each keyword.

If several professionals are targeting a particular keyword, it’s typically better to pass on it and find an easier one to go after.

Analyzing the existing search engine results for a keyword and determining how competitive they are is a whole different topic.

Most keyword research or rank tracking tools have some sort of built-in competition metric. However, some are more accurate than others.

If you’ve used the methods in this article to find keywords, your chosen competition metric will be much lower for your keywords than the easy ones that Google’s Keyword Planner gives to everyone else.

Beyond Google AdWords Keyword Research

Many people get keyword data from AdWords. But there’s other sources of keywords and search volume. We’re going to walk through a few of them now!

Bing

The newer Bing keyword research tool, although still in development, is a good place to check.

Sign in (or create an account)

Start with the basic settings. Bing’s keyword tool is a little different than Google’s. If set to 30 days, it give you the exact amount of search from those 30 days.

You can export the data.

Use the benefit of exact search numbers to judge long tail:

You can bring in Google AdWords data for a comparison of search volume metrics:

Now, let’s get a little more refined. Set the language to English (assuming that’s your target language) — and expand the date range to 6 months. This is going to give you a large data set to make good decisions:

Set to “strict mode” for basically Bing’s version of “exact match”

Wordtracker and SEO Book’s Keyword Research Tools

These versions are free. You can also register and pay for the pro versions.

Run the report in SEO Book to export CSV’s

Use the overall daily estimates for measure potential traffic across the entire web:

Wordpot

Wordpot is another source of keywords you may not have thought of.

It also shows definitions, synonyms, related words and associated words

YouTube Keyword Tools

Of course if you’re doing optimization for YouTube you’ll be interested in the search volume — but if not, this is still a fantastic tool very generating more keyword ideas.

Go to the bottom, click “add all 100” and then click export to CSV or the file of your choice

Also — let’s say you want to rank in youtube for “search marketing”.

1. Search for the term “search marketing” in YouTube
2. Find the top ranking video
3. Enter that video in the keyword tool — this give you the top keywords for that video

You can add these to the existing report or start over

Local Marketing Source — Local Keyword Tool

This is a handy tool for generating local keyword variations. It doesn’t give you search volumes, just a list of keywords.

Start with a zip code, radius and keyword(s)

Generate your list and export to CSV

Use Search Suggest Scrapers For Fast Keyword Ideas

The web collects data every day. Data that people are actually typing — they are telling us what they are looking for.

Search suggestions are a great place to look for words people are actually typing. This can often bring immediate content ideas as well as keyword prospects. The important part in this process is collecting as many ideas as possible.

Many SEO’s out there miss some of the most useful tools. Let’s walk through some now!

Soovle is the one scraper we recommend.

Soovle

There’s two ways we’re going to use Soovle. First, we’re going to use it as a general suggest scraper.

Set it is 15 engines. We want as many keywords as possible!

Start typing the root keyword surrounding the topic you’re researching

Use Scraper for Chrome to Key Them in a Google Doc

Who wants to cut and paste all of those words? We can get them in a Google Doc with a small tweaking to the Scraper xPath.

You’ll see the default results don’t include every keyword.

1. Eliminate the [4]
2. Scrape again (or hit enter)
3. Export the results to Google Docs.

Filter and sort the raw data

You’ll notice there’s a little extra stuff in there to clean up. There’s some data from others divs and duplicate keywords.

Delete the extra two rows at the bottom

1. Sort by keyword alphabetically
2. Now we need to filter duplicates! In cell B2 put this formula;=if(A2=A3,true,false)This will put true if the items are the same and false if they are different.
3. Then hover over the corner (wait for the cross to appear) and double click.
4. This marks every items as a duplicate or not. Before we do the next step, copy the true/false results and paste as values onto its self. This won’t visually change the true/false results – but will remove the formulas and leave just the words.
5. Sort descending by column B. This should put all the TRUE’s at the top. These are your duplicates.
6. Delete all the “TRUE” rows.

You’re left with about 100 of unique and diverse keywords!

Soovle To Catch Trends

Next, we’ll use Soovle to capture trending keywords if you want to create quickly and catch a trend.

1. Click on the StarThis will bring you to the trending words for the day.
2. Do a control-f in your browser. Search for words surrounding your niche
3. Then — enter THAT word back into the suggest scraper.

Use Google Trends

Google Trends is another not often talked about tool for keyword research, especially seasonal trends.

Additional Topic Ideas

Let’s say you’re a food blogger — and you like to do an article on red wine every once and a while. Wouldn’t you like to know some other topics you could attach to red wine? You can search this term in Google Trends and see the popular and helpful keywords around it.

You mightsee that a lot of people are searching for cheese related things around the same time they are searching for wine. This is somewhat expected, but there are many targeted keyword ideas you may not have thought of.

And you know these trend right along with red wine – so you could time your content to go with the seasonal trends a bit better too.

You can show more words and also do a CSV export

Ideas By Location

The location feature in Google Trends is great for targeting an audience in a particular location, with a parallel interest. This works great with seasonal items, or interests that fluctuate in search patterns through the year.

For example, let’s say you’re an ecommerce shop that sells grills etc. You might be interested in people searching for “outdoor party”. You know this search pattern matches the seasonality and interests of your customers.

Tracking Internal Site Search

Many site owners often overlook the most obvious place to get keywords. Your own website. We’re not just referring to keywords people use to arrive at your site. Rather, searches users are making within your site.

This especially works if you have a larger site — ecommerce or a large editorial site. You will have a lot of data to work with and many opportunities to learn about how people are using your site and keyword ideas you’d never think of otherwise.

We’re assuming you already have a search function in your site. We’ll show you how to set it up in analytics and use the data effectively.

Check How Your Site Does a Site Search

You’re going to need the URL parameter your site uses to track site search.

For example, the site we’re demonstrating this on, looks like the following;

See the search parameter?

/search.php?search=search+example

where “search example” is the keyword typed.

The parameter is “search” (hint: if in php its after the question mark)

Set It Up in Analytics

Head over to your analytics profile.

Go to the Admin area. Click on Profile settings

In profile settings, you’re going to scroll down to “Site Search Settings”

Type the query parameter we found in the first step in the field. Also, select to strip query parameters from the URL.

How To Use The Data

We’re going to show you how to best access this data for keyword research. We’re going to do this within analytics but you can also export the data for further use.

First – View the site search keywords

If you’ve never done this before, its best to zoom out at least six months.

We’re going to apply a few different filters to the data. (You could do this with advanced segments too for a more “permanent” filter).

This first filter says “show me words that get searched more often but then most people immediately leave my site. (You may have to play with the numbers specific to your metrics). These are the words you want to build new content around.

And may get something like this;

OR you can do sort of the opposite. Say “show me words that get searched a lot but where people stay on the site. Again, the exact number will depend on the behavior of people on your site.

Take Action

Either way, you’ll have a list of keywords you can take immediate action on.

• Highly searched words where people leave = create the content!
• Highly searched words where people stay = keyword research and optimize around those words.

Scraping Search Suggest From the Command Line

Extract Initial JS File

Google runs a script when computing its search suggest words. You can use the following URL in your browser, and in turn generate a JS file.

Create an .sh File

Simply copy and paste this code into your text editor
#!/bin/bash
q=$(echo “$1” | sed ‘s/ /%20/g’)
curl -s “https://www.google.com/s?sugexp=pfwl&cp=15&q=$q” | sed
‘s/\[/\n\[/g’ | cut -d'”‘ -f2 | tail -n +4

Save it as an .sh file. You can simply name it suggest.sh

Run The Code In Terminal (Mac)

Run the following command in terminal
./suggest.sh “marketing”
Which will produce the follow results;

• marketing jobs in wa
• marketing solutions
• marketing plan
• marketing solutions wa
• marketing jobs
• marketing strategies
• marketing drive
• marketing mix
• marketing internships

(note that some results may be localized).

Running Your Own Keyword Frequency Analysis

There’s many tools available out there to run a keyword frequency analysis. Some are free, some are paid. But there may be reason and benefit to having the control to run your own reports. We’ll show you one way to do this!

Create your .sh Code

Start by cutting and pasting this into your test editor;
#!/bin/bash

input=$1

IFS=$’\n’
# for keyword files…
if [ -e “$input” ]; then
cat “$input” | sed ‘s/ /\n/g’ >> tmp
for word in $(cat “$input” | sed ‘s/ /\n/g’ | sort | uniq); do
count=$(grep -c “^$word\$” tmp)
echo -e “$count\t$word” >> tmp_o
done
cat tmp_o | sort -hr
rm tmp tmp_o
# for web pages…
elif [[ “$input” =~ “https://” ]]; then
# check if links is installed. found solution at
https://stackoverflow.com/questions/592620/check-if-a-program-exists-
from-a-bash-script
type links >/dev/null 2>&1 || { echo >&2 “This script requires
‘links’ but it’s not installed. Aborting.”; exit 1; }
links -dump “$input” | sed -e ‘s/ /\n/g;s/\(.*\)/\L\1/’ | sed ‘s/\W//g’ >> tmp
for word in $(cat tmp | sort | uniq); do
count=$(grep -c “^$word\$” tmp)
echo -e “$count\t$word” >> tmp_o
done
cat tmp_o | sort -hr
rm tmp tmp_o
fi
Save this as a file — anything will work, such as keyword.report.sh

Run the Script

Open up Terminal in your Mac

You can run the script in two ways — either by file or by URL.

• By Filebash keyword.report.sh keyword.file.txt‘keyword.report.sh’ is the name of your .sh file and ‘keyword.file.txt’ is the name of your file with the content in it you’re analyzing.
• By URLbash keyword.report.sh https://www.quicksprout.comAgain, ‘keyword.report.sh’ is the .sh filename and then you just put the URL! Easy!

We bet you have a killer list of keywords and data now! Now your site is fast, crawlable, semantically correct, and you’ve got the best keywords. Time to target and create content that will attract links while you sleep.