Outline Shader

this is a shader to outline elments in the game using the sobel method while also allowing for the colour red to be shown.

initial idea:

At first, this Sobel outline was intended as an experiment to try to alter the art style of the game, due to aiming for a drawn art style as if the player was inside a news paper.

Secondary outline shader in the game that was occluded from the screenshot:

This shader was also layered on top of a shader I found from a tutorial which I did not include in the screenshot due to it being disingenuous otherwise given how close it was to the tutorial and the fact thatit would skew the proper effect of the shader. The secondary outline shader used world space normal and colour edge detection outlined them based on the strength of that normal detection, however I did remove the colour detection from it due to finding that it did not react well to the textures of the scene.

Sobel Explained:

sobel edge detection usually first starts with the input image being converted to a greyscale image due to working more effectively on. in sobel edge detection, a kernel is a small matrix used to convolute operations to proces and transform an image. There are two kernels, the vertical and the horizontal kernels, which are then used to calculate the properties of the sobel by changing their values, usually referred to as "weights".

Initial sobel experiment:

The initial version was very barebones and had a lot more emphasis on white on black rather than the black/white on grey that I was aiming for. another thing of note was that the lines were too thick while also being too light, which was not the look I was going for at all

Second Sobel Experiment:

After adjusting the sobel weights, I ended up with a better representation of the result I was looking for, however, it still posed the issue of how it will work with the detective mode shader.

Initial issue with the detective mode:

due to the detective mode working by replacing the material of the objects with the rather than overlay the colour natively in the fragment shader, the Sobel shader treated the red as a colour to fully outline

Shader adapted to fix the issue:

The fix for this issue was relatively simple, it consisted of taking the colour in the scene and if the colour texture is within a range of red, green, or blue(for flexibility) the shader returns the colour, subsequently if it does not detect the colour range specified, it returns the sobel edge detected colour which draws the outline