How to Test a Website || A Total Guide of Website Penetration Testing

In this detailed article we are going to discuss about how to test a web application step by step. This is gonna be a long article, and this will be a mega guide for web penetration testers and bug bounty hunters.

This article will cover all the categories of vulnerability and attacks a ethical hacker do while testing an website or web application. We are giving guarantee that these methods will leads to a vulnerability on a website, but it will give us a very good level of knowledge how the web penetration testing works.

Table of Contents

1. Mapping Contents of a Website

Scout Visible Contents

• First we configure our web browser to use our favorite proxy intercept or spidering tool. For this we recommend BurpSuite but WebScarab is also a good alternative. to passively spider the site and monitor and analyze website’s content through the proxy.
• Sometime it is very useful if we configure our web browser and use an extension like ZAP (OWASP Zed Attack Proxy) to check and parse the HTTP and HTML content processes.
• We can browse the website manually in a normal way. Visit each page and every link. We can try submitting forms and check it’s processing. Browse the website or web application with JavaScript enabled and disabled, and with cookies enabled and disabled.
• If the website have user registration then we have or can create a login account to test it’s protected functions.
• When we browses the website we need to aware about the requests and responses passing through our proxy intercept tool to understand how the data is being submitted and how the client is controlling the server-side application.
• We need to review the sitemap generated by our proxy intercept tool by the passive spidering. We look for those functionality or content that we have not checked through our browser.
• When we finished manually browsing and passive spidering, we can use our spider to continue actively crawl the website. This might sometimes uncover additional content that we overlooked when working manually.

Information from Public Sources

• We can use search engines and archives (like Wayback Machine) to identify to identify what content the website indexed and stored.
• We should use search engines advanced options to get more improved results. Like on Google we can use site: to get all the content for our target, link: to get other sites linked to our target website. Sometimes we can get old removed contents and get additional information.
• If we get any names, e-mail address or phone numbers then we can search them on search engines. We specially focus on the forums if the target asked about any technical detailed there. That way we may get some information on infrastructure.
• We should review any published Web Services Description Language (WSDL) files to generate a list of function names and parameter values potentially employed by the website.

Playing with Hidden Content

• First we run some tolls like DIRB and Gobuster to find hidden files and directories by using brute-force attack on our target website. Bigger word-lists will increase the chance of success.
• Know how the website/web application handles requests for non-existent items. We need to create some manual requests for known valid and invalid resources and compare the website’s response to establish an easier way to know when an item doesn’t exists.
• We should create a list of all known files, links and common file extensions. This will help us to guess the hidden files and directories. For an example if there are pages called AddDocument.jsp & ViewDocument.jsp then there is high chance to have EditDocument.jsp & RemoveDocument.jsp.
• We should review all client-side codes to find any clues about hidden server-side codes or contents. We should check in HTML comments and disable from elements.
• We should understand the patterns from the client-side to server-side process.

Find Default Contents

• To know about the web server we can run Nikto against it to detect what kind of server running. This may find any default or well known in the server. We should use Nikto’s options to increase it’s effectiveness. Like we can use –root flag to specify a directory to check default contents, -404 flag used to specify a string that identifies a custom ‘Not Found’ page.
• We should verify any juicy findings manually, it might be a false positive.
• We request the server’s root directory, specifying the IP address in the Host header, and determine if the application responds with any different content. If it happens then we need to again run Nikto against the IP address as well as the server name.
• We should request to the server’s root directory, specifying a range of User-Agent headers.

Identify Identifier-Specified Functions

• We need to identify any instances where specific web application functions are accessed by passing an identifier of the function in a request parameter.
• We have discovered the contents we do same to know the mechanism is being used to access individual functions. Like if the website uses parameter containing a function name, first we need to determine it’s behavior when an individual function is specified, and we should try to establish an easier way to identify when a valid function has been requested. We should compile a list of common function names or cycle through the syntactic range of identifiers watched to be in use. Also we can automate this task to identify valid functions as quickly & easily as possible.
• If need & applicable we can compile a map of web application contents based on functional paths unlike the URL’s, showing all the founded functions and the logical paths and dependencies between them.

Testing the Debug Parameters

• We need to find some website’s pages or functions where debug parameters are hidden, like debug=true. May be we find them in functions like login, search and file upload or download.
• We should make a list of common debug parameters like debug, source, test hide and their common values like true, yes, no and 1. On POST request we should check by submitting permutations of each name or value pair to each targets. Try them on both in URL string and request body. We can automate this by using BurpSuite’s Intruder (cluster bomb attack) to make a combination and permutations of two payload lists.
• We also check the applications response for any anomalies that may include that the added parameter had effect on the websites processing.

2. Analyze the Website

Scan the Functionality

• We need to identify the main functionality of the website. Also need to indicate the actions that each function is created to perform when it is used.
• Find out the core security mechanics in the website and how it is works. For an example we need to check how it works for authentication, season management and access control and the functions that support them. Like user registration and account recovery.
• Know more about the peripheral functions like redirections, external links, error messages, admin and login functions.
• We should look for any functionality that separate from the standard GUI activity, parameter naming, or navigation mechanism used in all the places in the website. We should check these one by one for an in-depth testing.

Look Inside the Data Entry

• We need to see the all different entry points that accept user data inputs into the web server. Also check the URLs, query sting parameters, POST data, cookies and other HTTP headers processed by the website.
• We inspect customized data transmission or encoding mechanics used by the website. Like a nonstandard query string. We need to understand where the data is being submitted encapsulates parameter names and vales.
• We need to identify any out-of-band channels through that user-controllable data or any other 3rd party data is being introduced inside of the web server’s processing. Like an web mail function will process and renders messages received through SMTP.

Know the Technologies Used

• We should identify each of the different technologies used in the website or web application. We will look for all the client-side technologies like forms, scripts, Java applets, cookies, ActiveX controls and flash objects.
• If possible we should check the server side technologies used on the server-side, like scripting language used, application platforms, and interaction with back-end components, like databases and e-mail systems.
• We also check the HTTP Server header returned in website response. Also we check the other functions identifiers contained within custom HTTP headers or HTML source code comments. Here different area of website might be used different back-end components so different banners may we got.
• We need to fingerprint the website using WhatWeb tool.
• We should check for interesting script names and query string parameters that may be from third-party code used. We can find them on Google using the inurl: qualifier to find any website using the same scripts and parameters.

Mapping the Attack Surface

• We need to determine something about the internal structure & functionality of the server-side and need to know what kind of mechanism running in the back-end. The idea should taken from the visible side or client-side perspective. For an example if a function stores items in shopping cart then it is most likely to be interacting with a database.
• For every functions, we should identify and lists the common vulnerabilities associated with per function. For an example, file upload functions may be vulnerable to path traversal. User chat and forums may vulnerable with XSS. Contactus functions may be vulnerable with SMTP injection.
• We should calculate a attack strategy, by keeping in mind the most serious vulnerable-looking functionality. We should use our plan to guide the amount of time and effort for each of our targeting sectors.

3. Testing the Client-Side Controls

Inspecting Data Transmission from Client

• We should look for all the parts where the website or server using hidden from fields, cookies and other URL parameters are used to transmit data from the client.
• We know the purpose that the item plays in the website’s logic, based on the context in which it appears and on it’s name and value.
• We modify the item’s value in a logical way, that will associate to it’s role in the website’s functionality. Detect if the website process arbitrary values submitted in the field if this fact can be exploited to interfere with it’s code’s logic or subvert any security controls.
• If the website transmit opaque data through the client, we can attack this in many ways. If the item is obscure, we may need to de-cypher the obscured algorithm and then submit the arbitrary data within the fuzzy item. Even if it is securely encrypted we may be able to replay the item in other controls to interface with other logics.
• If the website uses ASP.NET ViewState, then we test to confirm if this can be tampered with or whether it contains any sensitive information. We can use the ViewState analyzer in Burp Suite to confirm if the EnableViewStateMac option has been enabled, that means the ViewState‘s contents can’t be modified. We also carefully decode ViewState to identify any sensitive data if it contains. We should modify on of the decoded parameter values and re-encode and submit the ViewState. If the server accepts the modified values, we should care the ViewState as an input channel to introduce arbitrary data into the website’s processing. We perform the same thing on the data contains as we should request for other parameters.

Testing Client-Side Over User Inputs

• We need to find any cases where client-side controls, like length limits and JavaScript checks are used to validate user input before it’s submission in the web server. These controls can be bypassed easily, because we can send arbitrary request to the server.
• We need to check affected input field in turn by submitting input that would be blocked by the client-side controls to verify whether these are replaced on the web server.
• If we can bypass client-side validation that doesn’t means that there is a vulnerability. We should check carefully what validation is performing. We need to be confirm that the website is relaying on the client-side controls to protect itself from malformed input. We also need to confirm that there any exploitable conditions is available that can be triggered by the malformed user input.
• We need to review each HTML form to identity any disabled elements, like faded-out (Not clickable) “Submit” buttons. For example <input disabled=”true” name=”products”>. If we find anything we need to submit these to the server with the forms other parameters. We can use an automated proxy rule to automatically enable disabled fields. Like Burp Proxy’s “HTML Modification” rules.

Testing Browser Extension Components

Understanding the Client-side Operation

• We should setup a local intercepting proxy like Burp or WebScarab to check the client technology and monitor all traffic passing between server and client. If data is serialized, we should use a de-serialization tool, like Burp’s built-in AMF support or the Dser Burp plug-in for Java.
• We need to check the steps through the functionality available in the client. Also need to detect any potentially sensitive or powerful functions, using some tools like Burp.

De-compile the Client

• We need to identify applets (small apps) used by the website. We took for any of the following file types being requested via our intercepting proxy, like .class & .jar, for Java, .swf for Flash and .xap for Silverlight. We can also look for applet tags within the HTML source code off application pages.
• We also all requests made to the applet’s methods from within the invoking HTML, and detect if data returned from the applet is being submitted to the server. If the data is encrypted then we applet to obtain it’s source code.
• We can download the applet byte code by entering the URL in our browser and then we can save it on our local disk. The bytecode file’s name is specified in the code attribute of the applet tag. The file will be located in the directory specified in the codebase attribute if this is present. If not then it will be located in the same directory as the page in which applet tag appears.
• We should use some appropriate tool to de-compile the bytecode into source code. For an example we can use different browser extension to de-compile. Like Jad for Java, SWFScan and Flasm/Flare for Flash, .NET Reflector for Silverlight. If the applet is in a JAR, XAP or SWA file then we can unpack it using any archive extractor tool like WinZip or 7zip.
• We check the relevant source code to understand how it works.
• If the applet contains any public methods that can be used to perform the relevant confusion on arbitrary input.

Debugger Attachment

• On a larger website or a larger client-side web application it is very difficult to de-compile all the applets and modify them and repackage them without having lots of errors. So if we attach a run time debugger to the process. Javasnoop will work perfectly on Java. Silverlight Spy is a free tool to runtime monitor silverlight clients.
• We need to locate the main functions and values the website employs to drive security-related business logic and place breakpoints when our targeted function is called. We modify the arguments to return value as need to affect the security bypass.

Testing ActiveX Controls

• We need to find any ActiveX controls assigned by the website. An ActiveX control is a control that using Microsoft ActiveX technologies. Here we need to look for any .cab file types being requested via our intercepting proxy, or we should look for object tags under the HTML source code in the website’s page.
• Normally it is possible to subvert any input validation performed within ActiveX control by attaching a debugger to the process and directly modify data being processed or altering the program’s execution path.
• We need to test any ActiveX controls for vulnerabilities that could be exploited to attack other users of the application. We can modify the HTML codes used to invoke a control to pass arbitrary data to it’s methods and monitor the results. We can use COMRaider to test some basic fuzz testing of ActiveX controls, like buffer overflows.

Testing the Authentication Mechanism

Understanding Authentication Mechanism

• First we need to find out all the authentication technologies in use, like login, user registration, forgot passwords, forms etc.
• If the website doesn’t have an automated self-registration then we need to check if any other means exists of obtaining several user accounts.

 Testing Quality of Passwords

• We need to check for any password rules or minimum quality of *** passwords.
• We should attempt to try with some various kinds of weak passwords, establish the rules actually enforced. If possible we can try dictionary based attacks.
• We have to use the self-registration and set a strong and complex password (using upper and lower case letters, numbers and typographic characters and more then 13 characters). Then we try to login using different variations of the passwords, by removing last character, switching upper and lower case and removing any special characters. If any kind of these login attempt is successful then we continue experiment systematically to identify what kind of validation is actually being performed.
• We should check password quality rules and the extent of password validation. We also need to check the maximum character limits of passwords, and if it stores in server more then it without validation.

Inspecting for Username Emuneration

 

Source: Read More