Getting Started with Reverse Engineering

Course Overview
Hi everyone! My name is Dr. Josh Stroschein, and I want to welcome you to Getting Started with Reverse Engineering. I am an assistant professor in cybersecurity at Dakota State University, senior security researcher at VDA Labs, and a malware analyst for Bromium. The ability to reverse engineer binary code is an essential skill for anyone responsible for software security, performing malware analysis, and even plays a significant role at performing penetration tests and incidence response. In this course, we are going to learn how to quickly and effectively reverse engineer this binary code to determine program characteristics and behaviors. Some of the major topics that we will cover include the essential elements of modern CPU architecture and assembly language, native code and binary file formats, such as the portable executable file format, and techniques to enrich your reverse engineering efforts, and, finally, to gain proficiency and the tools and techniques associated performing both static and dynamic analysis. This will include both IDA Pro and WinDbg. Throughout this course, you'll be able to follow along through comprehensive demonstrations and apply yourself through hands-on labs. And by the end of this course, you will have the knowledge and skills necessary to reverse engineer native code binaries. Before beginning the course, some basic familiarization with assembly language will be helpful but certainly not required. I hope you'll join me on this journey to learn how to reverse engineer with Getting Started with Reverse Engineering at Pluralsight.

Introduction
Welcome to Getting Started with Reverse Engineering. My name is Dr. Josh Stroschein, and I'll be your instructor for this course. This course is designed to introduce you to all the key skills, components, and tools to become an effective reverse engineer. In this course, we'll be focusing on native code and the Windows operating system. There are many additional tools and techniques that we'll discuss along the way. This course will begin with an overview and the importance of why you should learn to reverse engineer. We'll then get into the basics of assembly and study CPU architecture, understand the difference between mnemonics and operands, and the differences in reversing 32- and 64-bit code. We'll then look at binary file formats and how working with native code can differ from working with interpreted languages. This will provide us with the opportunity to study the PE, the portable executable file format in fairly great detail. Next, we'll learn how to use debuggers as this course will introduce you not only to static analysis tools such as IDA Pro, but also dynamic analysis tools through the use of the WinDbg debugger. This will not only get you familiar with the key concepts and basic functionality, but also allow you to use debuggers effectively when reversing native code. Practical assembly will be next, and this will give you the opportunity to begin to decipher the output from our reversing tools to trace program functionality and answer that question, what is this program up to? Finally, we'll get into reverse engineering and leverage everything we've learned to add in IDA Pro. This will be an exciting course that will introduce you to everything you need to get well on your way to reverse engineering. And then it's up to you to explore the different paths that these skills can take you down, such as software exploitation and vulnerability research, and malware analysis. I'm excited that you're here and ready to begin. Let's go.

Assembly Basics
Now that we've discussed some of the theory and important reasons for reverse engineering, let's get into some of the technical details and discuss assembly basics. In this module, we'll begin by looking at the CPU, or central processing unit, in key areas of the architecture for the reverse engineer. We'll then look at the assembler and program statements to understand how programs are created and how they're eventually mapped into memory for execution. Next, we'll study the Intel instruction set architecture, or ISA, to gain a deeper understanding on how 1s and 0s are actually executed by the CPU. We'll then look at essential instructions and begin to construct a solid foundation for low-level reverse engineering. We'll then build upon that by looking at bitwise operations, endianness and size. And that will wrap up this module. Reverse engineering is a very broad and complex area of study. It is oftentimes not documented, or at least not documented very well, can be very ambiguous, and require vast amounts of research. This course is designed to focus on the essential elements and to get you effective at reverse engineering. Keep in mind, though, that this is just the beginning. As a reverse engineer, the learning never stops. I hope you're ready to begin. Let's get started.

Working with Native Code
In this module, we'll be taking a deeper look at working with native code. This is an important module as it sets the foundation and provides you key understanding into how a lot of our reversing tools actually work. In this module, we'll begin by looking at the portable executable or PE file format. We'll then take a look at the Windows Application Program interface or API. We'll follow this up with a demo, which allows you to get hands-on with a couple of PE parsing tools. And then we'll end this module introducing and going through our lab.

Using Debuggers
Debuggers are a core tool in any reverse engineer's toolset. In this module, we'll be taking a look at the basics of debugger functionality and how they fit in the broader spectrum of dynamic analysis. We'll then look at debugger commands, allowing you to become familiar with the basic operation and how you can use your debugger. We'll specifically focus on the WinDbg debugger. Next, get hands-on, and I'll provide a demonstration how to use WinDbg in order to analyze a sample program. In addition, you'll learn how to compile that from source and even generate debug symbols. Finally, we'll have a challenging lab, and this will allow you to get hands-on and analyze program behavior. A lot of great information awaits. So let's get started.

Practical Assembly
In this module, we'll be discussing what I call practical assembly. This will give you the knowledge to effectively trace through disassembled programs. In this module, we'll be talking about functions and the stack frame and how calling conventions impact function calls. We'll then have a demonstration in which I'll show how you can use the WinDbg debugger in order to view the call stack and identify each function stack frame. Next, we'll take a look at conditionals and control structures. These conditionals and control structures form the foundation for the different paths of execution within a program and are crucial to be able to identify, analyze, and follow. This will be followed by another demonstration in which we'll go through some examples of tracing this branching logic using our debugger. Next, we'll talk about arrays as arrays are a very common data structure used to store related information. Arrays can oftentimes be confused with simple local variables, and it is valuable to differentiate between the two when reverse engineering. We'll end this module on our lab and provide you an opportunity to trace through a more complex program, applying everything that we've discussed not only throughout this module, but really throughout this course. This is another important module and, after its completion, will allow us to begin working with our next reversing tool, IDA Pro.

Reverse Engineering
Now it's time to apply everything we've learned throughout this course and introduce IDA Pro. In this module, we'll begin by looking at the basics of IDA Pro and have a demonstration showing you how you can load your first binary. Next, we'll discuss strings and APIs, as well as a demonstration that shows you how you can leverage those in order to effectively trace through program behavior. We'll then deepen that understanding by discussing strategies for further tracing program behavior and follow that up with a lab in which you'll be reverse engineering an unknown binary. This will take your reversing to the next level. I hope you're ready, let's get started.

Conclusion
Excellent work! You've made it through this challenging course. In this final module, we're going to recap everything that you have learned and how that has made you a more effective reverse engineer. We'll also discuss ways in which you can continue to level up your skills, to gain familiarization with additional tools, to deepen your knowledge of reversing workflows and methodologies, and discuss resources that can help get you to that next level. And, finally, we'll end with a few thoughts on reversing in general.