All language subtitles for 005 OSINT Tools (OBJ 2.1)

-: Open-source intelligence tools are used during

the reconnaissance phase to find actionable intelligence

from various publicly available sources.

This intelligence can help the penetration tester to

be more precise and targeted in their attack development

and future exploits during their engagement.

Because OSINT is publicly available,

you're not gonna raise any alerts

at your target organization while you're gathering

information using only these OSINT tools.

Now, when working with OSINT,

you can collect information from any public websites

including the target's own website.

You can also check the WHOIS database to find

out who owns or operates a given domain,

as well as querying the public DNS servers to get copies

of the target organization's DNS records.

This can include all of their subdomains, email servers

and other softwares and service technologies

that are being associated with their domain name.

Now, job postings, blogs, news articles,

and social media are also useful

in gathering your intelligence.

From a technical perspective,

inspecting the target organization's

SSL and TLS certificates with the registry

authority can also be a useful form of OSINT.

Now, in this lesson, we're gonna take a quick look

at some popular OSINT tools that are covered by the exam.

For the exam, you do not need

to know how to actually use these tools

and all of their command line options,

but you should know why you would use each tool.

And in the basic case of OSINT,

and then more specifically what type

of OSINT information you could get from these tools.

Now, the tools we're gonna cover include Metagoofil, FOCA,

The Harvester, Shodan, Maltego, Recon-ng, and Censys.

It's important to remember that

open source intelligence is not trying to connect

to our targeted servers.

Instead, we're trying to find out information

about them by connecting to other servers

on the internet, or by searching elsewhere on the internet.

The first tool we have is known as Metagoofil.

Metagoofil is a Linux-based tool that can search

the metadata associated

with public documents located on a target's website.

This tool relies on the Python scripting language

to locate metadata with a different types of files,

including Microsoft Word, Excel, and PowerPoint

as well as OpenOffice's Writer, Calc, and Base files.

Metadata is not the data itself that's contained

within the file, though.

Such as the report you wrote for your college English class,

but instead it's data about the data in that file.

For example, this video consists

of all the ones and zeros that make

up the images on the screen that you're viewing

and the words you're hearing.

But the original video file also contains metadata

about the video file itself.

This metadata provides information such as the author,

the company who created it, title, and subject.

Other kinds of metadata can also be included in the file.

Such as how much time is spent editing a Word document,

the number of words in that document,

or the number of changes or revisions made to that document.

To use Metagoofil, you're gonna enter the command Metagoofil

and the appropriate options.

For example, if you wanted to search the Apple website

for any doc and PDF files, limit that search

to the first 200 documents found and only download up

to 50 of those files each time,

and then create a working directory where

you're gonna save those 50 files to,

and output the results to a file,

you can do that by using the command

Metagoofil dash D

apple.com dash T

dot comma pdf

dash L

200 dash N

50 dash O

Apple files dash F, and then press the Enter key.

Metagoofil can run on any system

because it is run by the Python interpreter.

So you can install it on Linux, Mac OS, or Windows Systems.

The next tool we have is called FOCA,

which stands for the Fingerprinting Organizations

with Collected Archives. FOCA's used to find metadata

and hidden information

and documents that we collected from that organization.

If our target organization is Udemy, for example

we might go onto Google and find out everything we can

about Udemy by finding Word and Excel documents, pictures

and anything else that company may have put

out onto the internet.

We collect this information and then we run it

through FOCA to find all the metadata

things like GPS locations, authors' names, email addresses

internal codes and things like that.

We can then use this as part

of our social engineering or to base our future exploits on

if we determine the software versions that were

used to create those particular files.

In recent versions of FOCA,

they've also added the ability to use Google, Bing,

and DuckDuckGo to find downloadable files

for you to analyze. Just like you can with Metagoofil.

Now, the nice thing about FOCA is

that it uses a graphical user interface.

And this makes it easier to work

with than the command line Metagoofil tool.

Unfortunately, though, FOCA does not work on Linux.

So you have to be running Windows if you want to use FOCA.

The Harvester is our next tool.

And it's a wonderful program for gathering emails,

subdomains hosts, employee names

email addresses, PGP key entries,

open ports, and service banners off of different servers.

As you can see, The Harvester is gonna start

with some of the basic open source intelligence

things like emails, subdomains hosts, employees

and things like that. But then we could take it further

by scanning it enumerating things like PGP key entries,

open ports, and banner grabbing.

Conveniently, The Harvester does come installed

by default inside of Kali Linux.

Now, The Harvester is pretty easy to use

once you learn its syntax.

To open up The Harvester,

go to your command line

and then simply enter The Harvester and hit enter.

And the command line options will actually show up

and tell you how to use it.

For example, if I wanted to search

for all the email addresses for a domain

like 'udemy.com' and then limit my results

to the first 250 that I find using Google, I can do that

by entering the command,
The Harvester dash D

udemy.com dash L

250 dash B

Google, and hitting enter.

The Harvester has the ability to use multiple

search engines such as Google and Bing

to find this open source intelligence.

But it can also search social media sites

like Twitter and LinkedIn as well.

If you wanna do that,

simply replace the word Google after dash B

with a search engine or social media site you wanna search.

If you wanna check a digital certificate,

you can also obtain the certificate information

from Comodo's certificate search engine as well.

You can even conduct banner grabbing

in conjunction with using something like Shodan.

Another tool that has a lot of great capabilities

like the Harvester, is known as Recon-ng.

In fact, Recon-ng has much more capabilities

than The Harvester, because it uses a system

of modules to add additional features

and functions for your use.

For example,

there's a module to conduct WHOIS queries to

identify administrative and technical points of contacts.

There's another one to conduct PGP key searches

and another for social media profile searching.

Yet another, for file crawling of websites,

and yet another one

for DNS record searching and enumeration.

If you have a list of emails

for your reconnaissance efforts, you can also

use Recon-ng to check if those emails have been the victim

of a data breach previously

by checking, 'Have I been pwned?'

And if so, you might be able

to find those passwords available

as part of a password dump for sale over on the dark web.

Now, Recon-ng is a Python script

and it makes it cross-platform

and supported by Linux, Mac OS, and Windows.

This web reconnaissance framework does come pre-installed

in Kali Linux by default.

To start it up, just type in recon dash ng and press enter.

Then it's gonna pop up

and you'll be able to see all the commands on the screen.

If you wanna try it out for yourself,

go ahead into your Kali virtual machine

and enter recon dash ng at the terminal prompt

and you'll be able to start playing with it.

When you load up Recon-ng, I recommend you

set up a new workspace to contain everything

that you're gonna be searching for and saving.

For example, you can do this

at the command line when launching it

by typing recon dash ng,

space dash W,

space Dion

and that's gonna create a new workspace called Dion

for me to use when I conduct my reconnaissance.

If you're already in the Recon-ng prompt though,

you can also just type in workspace, create, and the name

and it will actually create it inside of Recon-ng as well.

Once you do this, you're now gonna be inside

of this new text-based environment,

and you're gonna be able to use all

of the different features and functions of Recon-ng.

Now, similar to how metasploit is used

as an exploit framework,

and social engineering toolkit is used

for social engineering, Recon-ng is used

for reconnaissance and open source intelligence gathering.

It is a really complete all-in-one type of tool.

Next, we have Shodan, and Shodan is actually

a website that contains a search engine that lets us

find things like webcams, routers, servers,

and other devices that are considered part

of the Internet of things.

Both researchers and attackers both love Shodan

because of the wealth of information it provides.

For instance, today I found there are

over 10,000 webcams that are actually open to the internet

even though they shouldn't be.

Further, we can see there are 1500 things

using default passwords, making them very easy targets.

So, if we're targeting a company

we can actually search for things owned by that company

and see what they have opened to the internet whilst staying

at arm's length away because we're connecting to Shodan.

And Shodan is the one doing the searching.

Now, Shodan is looking for everything on the internet,

and so it's not targeting a particular company,

but it does add it to its database.

Think about, Shodan like Google for different devices.

In fact, Shodan is really good at finding things

like IoT devices, and many IoT devices are not

very well secured.

So as you come across things like webcams,

refrigerators, thermostats, or even alarm systems,

that may be a great way for you to get

into your targeted organization, because maybe

they just installed a new wifi-based security camera system

and you're gonna be able to use that

as your gateway and pivot point into their network.

The next tool we're gonna talk about is Censys.

Now, this is much likes Shodan

and it's another search engine.

The difference is it's only used

for finding hosts and networks

across the Internet with data about their configurations.

It has a really nice search interface

and it can build reports for us

and even has an SQL engine that we can tie

into if we pay for a subscription.

As a network defender, Censys can also be configured to

continually discover unknown assets on your network

and help you mitigate the risk of virtual machine

or cloud sprawl that can occur

in those type of environments.

The last tool we're gonna cover is called Maltego.

Maltego is a piece of commercial software that's used

for conducting open-source intelligence

and it helps us visually connect those relationships

between pieces of information.

For instance, if I gather a bunch

of emails or a bunch of social media profiles

of people who work for an organization,

I can start mapping them out with this tool

and figure out that if I wanna get to the CEO,

I first have to go through his receptionist.

Then I might find out that to get to the receptionist,

I need to go through her sister.

And if I figure out that her sister

likes the movie Twilight,

I might be able to start a conversation

with her and start connecting all those dots back up

and into the CEO's life.

Maltego is all about chaining from one person to the next,

or one server to the next, or one terminal to the next.

Whether you're doing email,

social networking or even machines themself,

you can collect all this information

and then map it out for exploitation using Maltego.

Maltego also has the ability to automate the querying

of public sources of data,

and then compare the data with other sets

of information to provide these commonalities

between different sources.

This data could be names, physical addresses, network

IP address ranges, phone numbers, email addresses

external URLs, DNS records, subdomains,

social media profiles, or downloaded files.

All of these results are graphically displayed

and all the links are visually created.

Similar to a detective trying to solve a murder case whose

as strings to tie together a bunch

of photographs on a cork board

so he can easily visualize the bigger picture.

That's what Maltego will do for you.