Kickstarter tutorial project feedback

[Gonan]

I would like to see how you should create your own source code in LUA, by starting with a new project, and what you should do to implement your changes to the existing starter code, so that you are never impacted by a new version, or project update.

Its very tempting to start tweaking the starter code, and save it back overwriting the original. For instance making changes to main.lua, means any updates to the main.lua made by a new leadwerks update, when applied to the project, with the project update facility, will mean remaking those changes by looking in the .bak files.

keeping a project uptodate with the latest version can quickly become a chore, especially if you are using the beta to get the latest features and fixes.

Managing this aspect of using leadwerks in a long running project, is my biggest negative experience, all I would need is a standard to follow for making user changes, that would eliminate or minimise the impact of this issue.

I'm sure someone else out there has found a way to avoid this from impacting their development.

[AggrorJorn]

The problem with this, is that you are spending time working around a problem that should not be happening in the first place. Since main.lua is always the start file (unless you work with entity scripts only) you will always face the problem of it being replaced with a new version.

Have a look at this 4 year old topic: 

 

 

[Josh]

Perhaps scripts should be skipped in the update process, and when you create the project it should be considered that you are "branching" the template.

[Rick]

In an ideal world when we decide to update it would show us any conflicts (like source control) and we get to decide what to do.

[Josh]

I don't want to add a new interface in the 4.x editor. This would also require that I dig into the OS on each platform and figure out how to retrieve file icons because people would ask for that next.

[AggrorJorn]

I think skipping lua files from updating a project is the easiest/quickest to implement. If a person wants an updated main.lua (or any lua file), model, material, he just needs to create a new project and copy it over from there to his original project. Shaders seems like something that most users will never touch and could still be updated imo.

[Josh]

So you're saying it should just update shaders, EXEs, and DLLs, and nothing else?

[AggrorJorn]

 

2 minutes ago, Josh said:

So you're saying it should just update shaders, EXEs, and DLLs, and nothing else?

Thats is what I would choose. Like in @Gonan suggestion, the most common issue is that the lua files have been changed by a user.  Both beginners and advanced lua users will tinker with main lua and the default scripts that come with a new project. 

[Gonan]

way back in the 80s when I was an IBM systems programmer, user exits were provided, which simply returned, by default, but would allow user supplied code to be substituted. Obviously this is done at each significant point in the supplied code.

eg.

main;

//initialise supplied code and setup internal_memory

up=addr(internal_memory)

call userexit01(up)

//start up

call userexit02(up)

// main loop

. call userexit03(up)

// main loop end

call userexit04(up)

//end

as user supplied code was changing a supplied module that simply returned, it never needed changing during a version change.

Also we never had to change main.

other Leadwerks users may have a more modern way of doing this.

An alternative is a optional user folder where modified versions of supplied code override the standard implementation. Leadwerks updates would not update the user folders. This would require some sort of path search , looking in the user folder before the existing folders, to be implemented in Leadwerks.

[Rick]

Main.lua is pretty brute force honestly. It should be setup to have callbacks at certain points (like Gonan mentions) and then the idea being we never actually touch Main.lua but instead work in the lua callbacks for our personal code. Include a Game.lua at the top of Main.lua and have that be where we define these callback functions. If we wish to make changes that have to do with the main loop we can do it in these. It's probably cleaner for new people as well to do custom things in this file vs Main.lua. This would allow us to hook into the main loop without changing Main.lua and having update issues. Each function should be checked for nil in Main.lua and not called or defaults provided if it's expecting a return result. This would future proof it and allow more hooks to exist over time and not crash if they aren't provided. The idea then is Leadwerks never updates Game.lua. We as the developer own it 100%. If we want to take the risk of changing Main.lua we still can, else we just use these hooks.

There would probably be more than these functions but the basics would be:

-- something like this that returns an object that configures the window
function WindowConfig()
	return { x = 0, y = 0, width = 800, height = 600 }
end

-- after the window/context/etc are setup
function InitGame()
end

function UpdateGame()
end

function PostRenderGame()
end

-- we define how to end the game
function EndGame()
	return false
end

 

[AggrorJorn]

I like your suggestion Rick. Still, I think lua files should not be updated in general. How often has the FPS player script changed throughout the years? This is what broke Project Saturn eventually. 

[Rick]

4 minutes ago, AggrorJorn said:

I like your suggestion Rick. Still, I think lua files should not be updated in general. How often has the FPS player script changed throughout the years? This is what broke Project Saturn eventually. 

Yeah that's true. The hook stuff should be there just for simplicity sake but not for this reason. I agree no lua files should be updated when updating a project. New projects should use the updated files though.

[AggrorJorn]

Regarding the topic subject: I used Udemy for the first time and I must say that I like their site a lot. I also have a pluralsight account from work, which reminded me of the course layout. So udemy would be pretty good to consider as well when I get back to a tutorial project. 

[Josh]

I really like the fact we can do this:

CreateContext(CreateWindow())
world = CreateWorld()
CreateCamera(world)
while window:KeyHit(KEY_ESCAPE)==false do
	world:Update()
	world:Render()
end

Do not like crazy hooks that take over your life.

[Rick]

38 minutes ago, Josh said:

I really like the fact we can do this:

CreateContext(CreateWindow())
world = CreateWorld()
CreateCamera(world)
while window:KeyHit(KEY_ESCAPE)==false do
	world:Update()
	world:Render()
end

Do not like crazy hooks that take over your life.

Except no game closes on pressing the escape key and games need ways to set the window size and do UI updating/drawing and other game loop stuff specific to the game that you can't think of. It's nice to look at the simplicity of this but this can't be used for any real game without modification which then gets into the issue at hand.

On a side note it's even easier when we don't have to see the game setup and loop and just have a handful of functions that get called.

[AggrorJorn]

All valid points Rick. On the other hand I agree with Josh: the game loop in its current state is what I always have found attractive about Leadwerks. Here is your base game loop and adjust the way you want it. 

I think making these kind of changes while Leadwerks 4.x is in the final stage would be a bit late. Maybe something to consider for Le5.

[Rick]

The current state of the Main.lua is pretty bloated honestly. It's strange Josh says he likes the simplistic one when the current one has a ton of other stuff in it. I generally comment out/delete/change the current one to my liking. I guess nothing prevents me from doing the hook idea. I'll try that out. It's just always nice when there is a general hook system so ideas/systems can be easily shared between people. That's where I go with this stuff. We still have a harder time sharing systems between each other because of this. If I have a system that needs an update and a 2D drawing part and an init sections, explaining where to put those in Main.lua is kind of a pain and could be harder for a new person. Explaining where to put them in defined hooks is a little cleaner.

The LE ecosystem is still the wild west. While there are good things about that, in my view there are more bad things about that. Unity's ecosystem is simply amazing. I'm not saying their product is good, I'm saying they've created a great ecosystem and their app store reflects this. It's much easier for programmers to create systems for others to use because they have a solid structure for that. They make design decisions to support that idea. LE does not and that's reflected in the limited workshop items and if most of them even work today without some getting into the details to configure/change to make them run.

In my view when you create a good ecosystem it keeps people on your platform. It's why Windows wins vs Linux in the desktop world. Much better and more user friendly ecosystem. Little to no fiddling around to get some driver to work.

These are reasons why "hooks" are important. Until that idea is introduced into LE, it'll always be the wild west and devs basically create their games from the ground up. Which as we see isn't that easy because of the amount of time and knowledge required to do so for a lot of different systems.

@AggrorJorn as you're seeing I'm sure there is little incentive to create some kind of system or framework to help people. Trying to integrate such things without any sort of help from the LE platform makes it difficult to integrate into everyone's projects.

From a business perspective having an ecosystem/platform is what every software company tires to do these days. Every system in Unity's app store helps expand Unity at no cost to Unity itself. Being a 1 man show in LE, Josh I would think you would want this because it basically outsources work for you. You can then pick the good ones to integrate into LE if you so decide. However, it's a feeling of losing control. I get that.

[GSFC_1]

Wow, this is a great thread. I feel dumb. 

 

[aiaf]

I like that simple game loop.

Unity is an example how to not do an engine imo , bloated ,frequently crashing editor.Did a game for a game jam with it and im not touching that ever again.

You create your specific systems for the game anyway and anyone will do it, le does the heavy lifting.

Even thinking of new users , is better to have something done quickly to build confidence and not learning what not framework from the beginning.

 

@AggrorJorn theres a site the pragmatic programmer bookshelf (https://pragprog.com/) have a look they have a section about becoming an author for them.

A small book about leadwerks would be awesome.

 

[Josh]

@Rick Sure, a huge ecosystem would be great. That's what paid Workshop items were supposed to do. But ecosystem effects only occur after a certain threshold of popularity is achieved. There is no technical design I could put out that would result in an immediate ecosystem because it's a business problem, not a technology problem. Read the book Crossing the Chasm if you want to understand why.

The last few years with Leadwerks 3-4 were a process of customer discovery. This is why I tried a lot of different things. Here are two things that worked really well:

• Steam (new customer segment found)
• VR support (business customers discovered, ability to differentiate based on performance)

Here are two things I tried that did not really work out:

• Steam Workshop
• Game Launcher

So as a result of all this I now have a much better idea of who the customer segments are. You probably have seen this before, the technology adoption curve:

I can come up with another one that is based on this specific industry. There are four market segments with different needs. These segments must be won over in order, or you can't effectively move onto the next segment: They correspond pretty closely to the segments above:

1. Technology enthusiasts (Innovators). Most of the forum regulars fall under this category. They are interested in the newest coolest thing and are risk-attracted.
2. Beginners (Early adopters). Most of the Steam users are this category, and I am sure you have seen their needs are completely different. They need good learning materials and examples.
3. Indie game developers (Early majority). These are people making games for profit. They are very risk-averse and want the safe bet.
4. Medium-sized game studios (Late majority). They care about customer support and the long-term viability of the company providing their technology. This is the only market segment I have not yet cracked.

The large game studios (Laggards) will always use their own in-house technology, and can't be won over.

My choice was to make new game templates and try to better support the second group (beginners) or to dive back into technology and re-attack the first segment again now that Leadwerks is a lot stronger than when I started. I feel like the first option was risky and could leave me over-extended while losing my base of support, so I decided to go back and design something new that would appeal very strongly to the first group.

By just making a new cutting-edge product I think I can win over the first two groups pretty effectively. Many decisions are being made for this purpose.

For the technology enthusiasts (most of you reading this):

• Substance support
• Visual Studio Code (brand association with something else that is new and cool)
• Vulkan (possibly)
• Forward renderer using new technique (possibly)
• Plugin system. I don't expect this will result in real ecosystem effects until about 10,000 subscribers are reached. I specifically made this decision because it gives you guys a way to tinker with the editor, which I know you will love to do.

I am doing these things in order to rally the customer base (you) and get new customers that are in this small but influential market segment.

For the beginners on Steam:

• API based on smart pointers makes scripting much easier.
• I looked at alternatives and nothing matches the simplicity of Lua.
• Good game templates will be a necessity.

So you can see why I put a lot of effort into getting Lua to work with smart pointers recently.

Now the question is how to penetrate the third group, indie developers, when they are so extremely risk averse? The answer is specialization. I intend to give them something they can't live without, at least a few of them. This is why I am investing heavily in revising the engine architecture. I'm not just looking for killer features, I am looking for killer attributes that effect the software on a system-wide level. Attributes such as:

• Performance
• Limitless scale

These aren't just a feature I can tack on, they are a guiding principle that is used to make decisions at every step in design. Basically, I am looking for things I can do that will give a small number of indie developers no choice but to use Leadwerks 5 because they cannot get what they need anywhere else. This is the only way to penetrate a market segment where everyone by default prefers to use the incumbent.

So that's the plan and I think it will work.

One really good thing we have going for us is it is very hard for Leadwerks to go out of business. We can stick around a long time looking for the right niche and developing technology. When Intel or Amazon launches a new game engine with lots of fanfare, the clock is ticking for them to achieve a certain goal or the project gets shut down. Because I don't have a lot of expenses I can afford to make a few mistakes and keep refining things.

[Gonan]

I agree with Josh, he must make decisions on the direction that leadwerks needs to follow, ensuring his business meets the needs of its users.

 

My original post also stands, and is not meant to conflict with Josh's plans. I was asking about how others managed their source code, and if there was a standard, or best practice to follow when setting up a new project. Having a tutorial on minimising the effort required to keep projects up-to-date.

Its not about main.lua, that was used as an example. What I really need is how to separate my user code from the leadwerks supplied code, so that it can be reintegrated after updating the project. I recently updated a project and was told over 400 files had been updated. Of couse the program no longer works, and would need a lot of effort to go back and reapply my code changes, just to get the program working again. That's why I call it a chore, as I have had to redo this work after each update.

I am asking the community if they have developed a standard approach to this problem. I am enthusiastic about leadwerks and the new features, but adding them into existing projects is too big a hurdle to cross knowing that I would need to repeat the process every few months. Imagine you had several projects written over the years, you would spend ever increasing time managing your codebase, and less time creating new work. As Josh says this is a user business decision, in how to manage what is an ever increasing legacy user codebase.

So please comment on your best practice, and how you minimise the rework required after project updates.

[Josh]

@Gonan I'm going to modify the way updating works to solve your problem. I think the danger of missing out on a script update is less than the danger of overwriting your modified files, and the API is quite stable so I think script updates are really optional at this point.

[AggrorJorn]

18 minutes ago, Gonan said:

So please comment on your best practice, and how you minimise the rework required after project updates.

This is my work method for the past 5 years:

• Try to stay away from default lua scripts were possible. If you need a script, lets say the fpscontroller.lua, make a copy and use that.
• Use source control like git or mercurial. Make sure you have no changes and then do a project update.
  • You can then easily identify changes in files that you do want to included (dll's, shaders)
  • You can then easily identify changes in files that you do not want (main.lua,  fpscontroller.lua) and simply revert the changes from your source control manager.
    • This way you also quickly find any changes to your lua files.

 

 

[Rick]

3 hours ago, Josh said:

There is no technical design I could put out that would result in an immediate ecosystem because it's a business problem, not a technology problem.

If you removed the word immediate then I don't agree with this statement. Nothing is immediate we all know that, that technology supporting the growth of an ecosystem is a technology problem. One of the reasons FB got really popular is because they created a way to make FB apps. They created a technical way to support an ecosystem. It's a mindset of "If you build it, they will come". If you give a technically sound way to extend things then those 2.5% innovators will build your engine for you (to some extent). 

 

3 hours ago, Josh said:

My choice was to make new game templates and try to better support the second group (beginners) or to dive back into technology and re-attack the first segment again now that Leadwerks is a lot stronger than when I started.

In my view it's a waste of your time to build templates. Let the 2.5% innovators do this for you. BUT, they need a standard way of doing this and they need your support and promotion (no technical needs here really just promotion stuff) of those things to gain credibility to the other groups. What could be an example of what you'd do around this? Well how about a more dynamic template system when someone creates a new game. That screen should be reading from some place online to list the templates available instead of local machine. There should be an official and community section of templates at the point of creating a new project. We should be then given a streamline way to create & publish these templates. When you create a means to do something officially, people create stuff for that.

In my view your 2.5% innovators are an extension of any company. Often they are free labor because they do things out of love and learning and not profit. You've generally not empowered those 2.5% innovators for a fear of giving up some control I would think. The innovators have been marginalized because they are all doing things their own way and get no promotion or help from a framework that guides how things should be done and how users access their work. This is all about technology. This is about standard systems of game template creation/promotion and plugin type systems and standard ways to hook into LE. All which don't exist.

 

3 hours ago, Josh said:

This is the only way to penetrate a market segment where everyone by default prefers to use the incumbent.

In my view Leadwerks selling point is it's simplicity. Period. Performance just has to be there and be competitive. It's a must have, not a selling point. Leadwerks attracts noobs because it's simple to use compared to the competition.

 

*Disclaimer: The above was just my opinion. I'm just acting as a sounding board

[Gonan]

1 hour ago, Josh said:

@Gonan I'm going to modify the way updating works to solve your problem. I think the danger of missing out on a script update is less than the danger of overwriting your modified files, and the API is quite stable so I think script updates are really optional at this point.

Thanks Josh, looking forward to trying it out. Are you allowing updaters to see the updates leadwerks have held back due to user updated conflicts. So that they could choose to integrate them themselves. As you said LE 4 api is pretty stable, so there should not be too many. Hopefully I can use a backup of a working project, and then reapply the new version of the project update.

LE 5 is going to be the start of a new set of changes, will the same update process apply?