Everyone makes mistakes. We try not to, but screwups happen anyway. We live and work in a world of ever increasing complexity. We humans, don’t get everything right 100% of the time. None of us. Given that, how we respond when we screwup is quite important. Ultimately, when you screwup, you have one fundamental choice. How are you going to respond? The screwup is over…you can’t go back in time.

Try to hide the truth?

Blame others?

Make excuses?

Nope. None of those.

My suggestion: Own it.

I’ll explain why in a moment, but first a story about one of my worst screwups. I hate this story.

20+ years ago, I was a nineteen year old kid starting out in an I.T. career. I worked in an office with ~40 people. It was my job to manage an email server. Pretty cool job for a 19 year old kid…right? I was young, eager and awfully green. When you are young and green in I.T. you have a lot of blind spots you aren’t even aware of yet.

One day, the email database on the email server (NT4 / Exchange 5.5) I managed got corrupted. That makes for a pretty bad day generally for the email server guy. Server down. No email for anyone etc. So, I started working to restore it. We had backup software that I faithfully checked each day. However, when I went to use it to recover the data, the restore failed. Uh oh.

After a long painful call with the backup software tech support folks, I learned that the type of backups we were doing were not actually suitable and I was essentially out of luck from a restore standpoint. Such a total noob mistake. At that time, I honestly had no idea. Some consultant had popped in before I was hired, setup the server and gone away. However, it was ultimately my responsibility to manage it, and I had a blind spot I was not even aware of that hit me and my employer hard.

By this point, it was after normal work hours. So, I did what every desperate Exchange server admin did back then. I called Microsoft for help. They helped as much as they could. We ran some low level database tools (eseutil and isinteg) that wiped out a bunch of corrupted database pages, but left me with a database that would mount. We saved some of the data, but not all of it. We finished up in the wee hours and I went home to grab a couple of hours of sleep.

I came in early the next day, and I spoke to both of the owners of the company I worked for. I explained what had happened and that a lot of email data had been lost. I explained that I had done everything possible to save as much data as could be saved. I sent an email out to the entire company. Oh the irony, here is an email about your email being gone. Some people did not seem to care that much. A few others were pretty mad. I did not blame the mad ones. I was mad at myself too.

Then something crazy happened that I will never forget. One of the owners was discussing it with one of the folks who was still mad. I could hear them across the hall. An odd thing happened. The owner got tired of the pointless complaining, and shut it down. He told the mad person that if some email they had was so important, they should have printed it out. It was their fault really. Wait, what? Printed it out? Ha! I guess when you own the place, you can say stuff like that and get away with it. Basically, I think he did this to get them off my case and stop the pointless complaining. It was amazing. It was totally undeserved. I sure was thankful for it.

Thankfully, it was the mid-90’s so email was less business critical back then than it is now. After a few days, people generally got over it and moved on. I doubt most of the people involved even remember it today, but boy – I sure do.

I’ve learned some lessons from that incident, and other screwups I have watched play out in the 20+ years since then. Here is my personal philosophy at this point on screwups, given what I’ve observed.

When you screwup, own it. Here is why…and a bit of how.

When you screwup and own it, people don’t question your character. Sure, you screwed up. That is bad and there may be consequences. However, you will probably live to fight another day. I went on to work for the company who’s email server I failed to properly protect for about five years total. Over the years, I earned the ever increasing trust of the owners I worked for. Why? In no small part I think because the owners knew even when it would be hard, I’d tell them the truth. I might not be perfect but I was honest and I was working hard to get better. As any business owner or manger knows, you can work with that.

Had I lied, tried to cover it up, or made excuses they would have seen right through it. I would have destroyed their trust in me. Ultimately, protecting that data was my job, and I had failed in this instance at that job. No pile of lies or excuses could change that reality. However, I was honest about my failure which left us both with a foundation of honesty and trust to build from.

Your character and credibility is ultimately what you build your life on. Own it by telling the truth – the raw, sometimes ugly, sometimes painful truth. People will know you are not perfect (hint: they already know that), and they will learn that you are honest, even when it is hard to be.

When you screwup and own it, you can learn from it and do better. Over the years, I took my failure seriously and became a more careful I.T. Professional. I worked to add diligently to my base of knowledge to avoid these sorts of blind spots. I learned to plan projects with the potential for failures and associated roll back / restore plans in mind. When I somehow ended up with responsibility for something I realized I did not adequately understand, I dove in and learned as much as possible as fast as I could.

When you don’t own it and you pretend or make excuses, you can’t learn from your mistakes. You are too busy investing energy in making excuses and pretending everything is fine. If you screwed-up, everything is not fine. Don’t pretend it is. Instead, face it, own it and challenge yourself to do better.

Invest your response energy in something productive, not in making excuses or hiding the truth. Over time, and multiple screwups your ongoing investment in either activity piles up. You either pile up war stories and lessons learned that have made you better, or you pile up excuses and incompetence. Choose your pile wisely, you’ll get to live with it.

When you screwup and own it, do everything you possibly can to make it right. Part of owning your screw-ups is doing everything you can to correct them. In the end, I was not able to bring all that dead email back to life, but I did do everything possible to save as much as possible. I think this is why that business owner stood up for me. He knew I had done all that I or anyone else could have possibly done. There was really no point in anyone giving me a hard time, so he shut Mr. Mad Guy down for me. He knew, I had learned the lesson. I wonder if he realized I’d learned it well enough to be blogging about it 20+ years later?

In my life, this experience helped me learn the importance of protecting data in a visceral way. Protecting data is serious business. That employer never lost a bit more data on my watch. I had learned the lesson. Since then, I’ve configured many backup systems for many clients and I’ve done countless successful restores. I take every single one I touch ultra-seriously. Might I some day lose data again? I guess anything is possible. I do everything possible I can think of to prevent it, but I’m also humble enough to realize I still may have blind spots (unknown unknowns – thanks Mr. Rumsfeld). Now I have the great benefit of 20+ years of experience protecting systems and data, so there are far less of those. I have also gained the benefit of a deeply emotional connection to how important that work is.

When you screwup and own it, the worst is over. Ultimately, sometimes you make mistakes you can’t entirely undo no matter how hard you try. If you’ve done all you can to fix it, and you’ve truly owned it, it’s over. There is nothing to be gained by beating yourself up. Fortunately, by owning it you also have not hidden your error or lied. Doing that only sets you up for trouble down the road.

So, my advice to you, most patient reader, is this: When you screwup, own it. By doing so, you will preserve your character and good name, you will learn from it and get better, you can sleep easy knowing you did all you could to correct it, and the worst will be behind you.

Here’s to avoiding screwups! When that fails, here’s to owning them well.

TL;DR:  If you are a parent who wants to better manage how your kids use the internet – go buy a Circle with Disney box.  You can thank me later.

Back Story: One night last month just before going to bed I decided to review my nine year old daughters iPad browser history.  We are not some sort of crazy – spy on everything your kids do – sort of folks.  In fact in retrospect, we may have been too permissive on this previously.  However, I do think as parents monitoring and managing how your kids use the internet is part of your job as a parent.  So, periodically I check the kids internet history.

It had been a while since I had done that on my daughters iPad.  On this night, I was horrified by what I found.  Earlier that very evening she had been watching some generally kid friendly stuff on YouTube.  However, looking back through the history you could see how her curiosity led her down a bad path.  That bad path ended with her on the web site of a hate group, “learning” about a very sensitive topic.

Waves of emotion washed over me – anger, sadness, and a lot of determination to do something to prevent this in the future.  I showed my wife the iPad history and I went and got my daughter.  She came down to our room and we sat on our bed together.  For the next half an hour or so, my wife and I unpacked the topic she was curious about, shared our feelings and beliefs on the subject and explained that she had gotten some really bad information from a really bad source.  After a long, emotional discussion – we prayed together and she went on back up to her room to sleep.

I felt so bad.  How could I – a dad who is I.T. Infrastructure Engineer for a living – not have done a better job of protecting her from some of the awful stuff online.  After all, I install enterprise grade equipment for corporate clients that filters out this sort of stuff.  My first inclination was to buy a smaller version of what I generally deploy for clients and install it in my home.  However, even smaller versions of that equipment are very expensive and honestly total overkill for home use.  Then I remembered a friend mentioning that he had purchased and deployed Circle with Disney on his home network.  From what he had told me previously – it was working great for him.  So – at  11:43 PM on Saturday night, desperate to take some sort of action to solve this, I fired up my phone and ordered a Circle device for $99 from Amazon.  My expectations were pretty low.  So much consumer I.T. equipment is total garbage.  However, I was determined to do something.

Progress: A couple of days later, the Circle box arrived and I unpacked it.  Setting it up was very easy.  It would be easy for those of you who are less technical as well.  If you are comfortable with using an app on your phone, and you know your home wireless network name and associated password – you can do it.   You simply add the Circle app to your phone, and follow the directions.  The initial process is very well designed and the initial connection process is very easy.  Within a few moments, I had the device on my home network.

Circle App – Home Screen

The way the device works is by discovering devices on your network, and associating those devices with either a particular person, or the house in general then associating each of those entities with some basic use controls.  In our house we ended up with a lot of devices either associated with my wife, myself, or our house generally.  Those profiles and therefore those devices don’t have any sort of filtering or controls applied.  If my wife wants to look at crazy stuff online – who am I to stop her?  Ha!  Seriously – the device just ignores traffic from those devices.  This is great.  The last thing I needed was a consumer grade device interfering with my work use of our home internet connection.

I then associated the devices my kids use to profiles I setup for them.  Circle has some great pre-built filtering profiles that were very easy to customize.  Currently, both of my kids are setup with minimally customized versions of the “Kid” profile.  I turned all of this on without telling either of them.  I wanted to see if they would even notice.  They noticed.

Circle App – Profile Settings Screen

Issues: Shortly after the next time my daughter went online, she noticed that she could no longer see videos from one of the YouTube channels she was watching previously.  Circle was forcing her YouTube views into YouTube Restrictive Mode.  So, as far as she could tell – these videos were just totally gone.  Being the extremely reasonable parent that I am – ahem –  I explained that I had added some filtering and I asked her to come down so we could watch the video in question together.  We made it about five minutes in, and after being subjected to a good bit of profanity and a lot of generally tasteless content – I told her we would not be unblocking this.  This was a catalyst for a good follow up discussion with her about being careful what content you consume online.  Being the generally reasonable daughter she is, and knowing there is no way on earth I would change my mind, she went back to watching kid appropriate stuff.  Bingo!

Circle App – Per Profile – Filter Settings Screens

A bit later my son came down.  He is a bit older than my daughter, and he has a good history of being very reasonable with what he watches online.  We have had a lot of discussions about this.  He was incensed because he could not watch some of the YouTube gamer videos he likes to watch.  Given the age difference, and his generally good history of managing what he watches well – I turned YouTube Restrictive Mode off for him.  However, I reminded him we could and would monitor his activities online.  Him knowing this encourages reasonable use.

The only other issue we have run into since we installed Circle about a month ago was with a site my son uses for school.  Circle was blocking it.  I simply went in and added the site domain to his custom filter and allowed it – problem solved – all from my phone within a few seconds.

Circle App – Per Profile – Custom Filter Screen

Unexpected Benefits: One other issue this has solved for us is device use near bedtime.  We have times of day set (different on school days and weekend) where the kids internet access just turns off.  The kids are welcome to read, draw, whatever but screen time is over for the day.  Previously, each night we would tell the kids “Time for bed…” and they would reply with “Four more minutes please!” or something like that.  Now they know – at 8:45PM on school nights the devices just stop being able to get online, so they plan accordingly.  Bedtimes around here have gotten a bit easier.

Circle App – Per Profile – Bedtime Screen

We also have a setting in place that only allows the kids to be online one hour per day – unless my wife or I add more time by hitting the reward button.  To be honest – we hit that button quite a lot.  The way we use this currently is to make sure they do their school work etc before we add more time.  So – they can come home from school and chill online for an hour if they want – then it’s time for them to get homework etc done before they ask us for more time.

Circle App – Per Profile – Time Limits Screen

Summary: The end result of deploying Circle has been nearly ideal.  The kids are protected from a lot of content that is not appropriate for them.  In addition my wife and I have dramatically better control of how they use the internet in general.  We are also well positioned to manage this in the future.  As they get older – we can easily open this up further and further.

Caveat Emptor: I have no illusion that we are perfect parents.  Perfect parents would have managed this whole thing better from the beginning.  My goal here has been to be transparent and share what I’ve learned in the hopes that sharing this will help others.  What works for our family may not work well for your family.  I totally get that.  However, I will humbly suggest that if you are a parent who believes that its part of your responsibility to manage your kids internet use, that this would be a great tool for you to consider.

Oh yeah – one last thing: You know what my little girl sees when she goes to hate speech sites now?  This:

Yay!

To the team of engineers and developers who built and manage the Circle platform: You all are awesome!  Great work!  Thank you!

Disclaimer: I have no idea if this post will ever be helpful at all to anyone else.  Sometimes writing something like this helps me make sense of my own thoughts.  Since I wrote it down, I thought I would share it.

A commonly repeated bit of wisdom is that it is a wise career choice to “Do what you love.”  I’ve heard this advice for years and at first glance, it seems good.  However, as you really dig into it, it can be difficult to figure out what it is that you really love.  What would it look like to actually do what you love for a living?

Is this what I really love?

I love to go to my Aunt’s lake house and ride the SeaDoo around the lake.  It’s one of my favorite leisure activities.  Should I do that for a living?  Is that what I love?  Thoughts like this go through my brain…  Perhaps I can make a living riding a SeaDoo.  Perhaps I could become a SeaDoo racer.  Do what you love – right!  YOLO!

Hang on a moment Mr. YOLO man.  Let’s pause and count the potential cost of doing that.  I might need to move away from my extended family to some warmer climate where you could do this year-round.  I might need to exercise a lot to get in great physical shape to race competitively.  It’s nearly impossible I would be able to earn much of a living doing this in the beginning, so for several years I would need to train to be a SeaDoo racer and work at another job to support my family.  My free time and a lot of my family time would be totally consumed by this.  Hmm.  This is sounding less awesome very quickly.

Once you look at what you think you love in the bright light of reality, the picture changes a bit.  I do enjoy riding the SeaDoo.  However, I don’t love it nearly enough to make all of the sacrifices I would need to make in order to make an actual career out of it.

Now what?  Give up?  Head back to the proverbial salt mine to spend the rest of my days doing something I really don’t love or perhaps even hate?  Nope.  Dig deeper. Here are a couple of things I have observed recently that have made me think differently about this topic.

Observation #1 – Olympic Swimmers

Recently we watched some of the 2016 Summer Olympics.  I was amazed watching the swimmers.  Think about what they did to even make it into that Olympic pool.  They exercised like crazy.  They ate healthy – probably extremely healthy.  The practiced over and over again, nearly perfectly, day after day for years upon years.  They sacrificed a lot of big things in their life only to make it into the Olympics.

Watching them, I sat and thought at first, “Wow.  That is cool. They sure are fast!”  Then I thought a bit deeper about what being that fast must have required and I thought: “You know what – these people are crazy!  Why spend so much of your life for so many years on end to become THAT good of a swimmer?  Who cares!”

I for one frankly do not care nearly enough about swimming to do that.  I would not be willing to invest even a small fraction of the effort that the person who came in dead last must have invested, even if you could assure me that by doing so I could be an Olympic gold medal swimmer.  It’s simply not something I care that much about.  Those swimmers must really love something about swimming.  They have paid a tremendous price to get to this point.  I don’t think it is a price anyone would pay, if they did not love it.

Observation #2 – Wonky I.T. Security Topics

Last week, somehow I came across something that peaked my interest in an I.T. security book called the Art of Memory Forensics.  So, I paid ~$50 and ordered it from Amazon.  It came in on Friday and I proceeded to give it and the great tool that it is written about a ton of my weekend free time.

Why?  Because memory forensics is awesome!  Well – at least I think it’s awesome.  It’s a tool that can help me do something I actually love even better.  It open up another possible angle to attack the problems that I wake up thinking about from.  It is a small piece of a grander puzzle that has had me fascinated for years.

If most of you were to read this same 858 page book, you would hate it.  You would be bored to death.  You would not be willing to invest a fraction of the time that I will happily invest on this topic.  You would probably not do it even if you could become one of the best memory forensics people in the world.  You would nearly die of boredom or confusion or perhaps both during the first few hours.  Why?  Because most of you don’t care at all about this topic.  You don’t care so much that I bet your brain would nearly refuse to focus on this for long enough to really learn much about it.  You don’t care about memory forensics in the same way I don’t care about swimming fast.

What I have learned

Why do I love topics related to Computer Security?  Honestly, I have no idea.  I just do.  Perhaps it is just what God put me on this Earth to do.

Looking back over my life, I can see a clear interest in this topic all the way back to when I was a kid.  I remember one year, my family was at the beach and somehow I ended up reading a bunch of Tom Clancy books.  Perhaps they happened to be at the house we rented for the week.  Spy stuff, military stuff, tapping undersea cables to gather intelligence on the bad guys – all of it seemed so fascinating!  I did not get nearly the amount of sun that my brothers and cousins did that year.

After hearing I was a Tom Clancy fan, my High School Principal pointed me to a book by a guy named James Bamford titled The Puzzle Palace.  I had never heard of the Nation Security Agency before this book, but I read every word.  Again – totally fascinating.

Fast forward my life story a bit and I ended up in an I.T. career.  As my career has progressed, I’ve always gravitated to areas that fit within the broad categories of Information Assurance / Information Security.  I love configuring firewalls.  Seeing an IPS alert on a blocked attack is an actual thrill for me.  I love well planned and well configured backup systems.  Quickly restoring data that was destroyed by a crypto ransomware attack knowing that the capability to do that mean the criminal will not get a dime of my client’s money makes me happy on the inside.  I imagine I love these things as much as those crazy swimmers love swimming.  It’s just IN me.  It’s what I actually love doing.

Could I do something else?  Sure, but I might not enjoy it enough to get really good at it.  For me, this is the area that is so fascinating that I will willingly invest my free time and personal money to learn even more.  For me this is a marker.  It’s a hint.  It’s an indicator.  It’s a pointer that points to what I must really love.

My $.02 worth of advice for you if you are wondering about what you really love.

Sometimes people struggle to figure out what they love.  If you can’t figure out what it is that you love, look at your life and ask yourself this question:  What is it that I am so fascinated with that I will happily spend my free time and my own money learning more about?  For some of you it is music.  For others, it’s real estate.  Perhaps for some of you it is cooking.  For others it’s helping hurting people put their lives back together.

Look for patterns.  If you’ve been fascinated with something for years and you’ve spent your own money and your own free time to learn more about it and/or do more of it – pay attention.  That might be your thing.

If you think you might want to make a career out of it, pause and run it through the sacrifice filter first.  Ask yourself “Do I really love this enough to sacrifice what would be required to become good enough at this to make a living out of it?”  Be warned – the sacrifice required will probably be even higher than you expect upfront.

If you are not willing to sacrifice enough to make a career out of something, that’s ok.  Perhaps whatever it is can still be a great long term interest for you.  For me it has been helpful to have thought of some things and then intentionally set them aside as career options.  Doing this frees you up just to enjoy them as interests without getting stuck thinking about a career move that you know deep down you are not willing to actually make.  Set them aside and over time move on to the next thing you identify.  Rinse and repeat.  Eventually, you might hit on the thing you really enjoy where the cost to actually do it fits with what you are willing to sacrifice.

Back to me for a moment…

Do I do exactly what I love 100% of my work time.  Nope.  I honestly doubt anyone does. However, I get to do enough of it that the sacrifices are worth it.

If I’m lucky, I’ll make it to my Aunt’s place this weekend to ride around on the SeaDoo.  However, when I get home and clean up, I’ll probably be thinking about some  I.T. security related topic while I’m in the shower.

If anyone reads this far – I’ll be amazed.  If you do, I sure hope you can find and do work you love as a career too.  If you want to read more on this topic, here are a few links to folks who have shaped my thinking on this.

• Michael Hyatt (podcast / blog posts / etc)
• Dan Miller  (podcast / blog posts / etc)
• Cure for the Common Life by Max Lucado (Book)

So – obviously I’ve not posted about learning to code in the last few weeks.  That is because, I’ve not been learning much.  My I.T. infrastructure consulting work has had me pretty slammed.  I hope to be back to learning to code more next week.  More then.

If you wake up and see IPs you support routing to China, it’s going to be a rough day.  Today – was a rough day.

• At 4:35AM EDT my network monitoring system alarmed that a clients site-to-site VPN connection was down between the clients office in NC, and our data center in Atlanta, GA.
• At ~ 6:15AM EDT I woke up and saw the alarm.  I immediately begin testing / collecting data.  It quickly became obvious that this was a routing issue.  Connectivity between some networks (Road Runner and several others) to our clients data center IPs was broken.  Curiously – traffic from Road Runner / Time Warner Cable was routing out to a router in Los Angeles, CA then dying.
• In order to open trouble tickets for a routing issue, you need trace routes.  So I collected several showing networks that worked and ones that did not – in both directions.  Then I opened tickets with Road Runner / Time Warner Cable (the clients ISP) and the data center (who provides us IPs as part of a BGP mix of bandwidth they maintain and optimize).
• After some additional troubleshooting while waiting to hear back on my trouble tickets, I noticed that a new BGP advertisement which included our IPs was published at nearly the exact same time that the site-to-site VPN failed.  I’ve sanitized the screen shot to protect the innocent (my client) and the guilty (a Chinese ISP).  The red blocks contain IP details I’ve intentionally removed.
  
• After some troubleshooting we were able to determine that a Chinese ISP had published a bogus BGP advertisement. The Chinese ISP wrongly advertising the a /20 block of IPs (which included some of ours).  They actually own a /20 that was one character different from the block they advertised.  It appears they simply made a typo somewhere and caused all of this.
• Our data center NOC team reached out to the Chinese ISP NOC to see if they could get them to remove this wrong advertisement.
• At 10:25AM EDT our monitoring system recorded the site-to-site VPN coming back online.
• When I arrived at the client site (where I was scheduled to be today anyway) – I tested and the bogus BGP advertisement had been removed.

So – what is the take away from this?  What can be learned?  Here are a few things – several of which I knew intellectually previously and I know at more of a gut level now.

• False BGP advertisements can create a real mess.  I knew this previously – but it never impacted me as harshly as it did today.  Want to read more on how bad this can be – check out the BGPMON blog here: http://www.bgpmon.net/blog/.
• It seems some ISPs filter or manage BGP more carefully than others.  For example – Level 3 never seemed to be effected by this bogus BGP update.  Time Warner / Road Runner apparently accepted it nearly immediately.  I’m no BGP guru at all – but wow improvement is needed here.
• In the future before I open a routing issue ticket, I’ll take a look not only at trace routes, but also at BGP advertisements.  Huge thanks to Hurricane Electric for a great looking glass tool that ultimately helped me get to the bottom of this.