hate these ads?, log in or register to hide them
Page 121 of 122 FirstFirst ... 2171111118119120121122 LastLast
Results 2,401 to 2,420 of 2421

Thread: <xml><thread isTerrible="true" isUnderstandable="false" /></xml> - Coding help thread

  1. #2401
    Donor erichkknaar's Avatar
    Join Date
    April 10, 2011
    Posts
    9,126
    Quote Originally Posted by Frug View Post
    Quote Originally Posted by elmicker View Post
    Quote Originally Posted by erichkknaar View Post
    I assume your comment about Apache is really pointing at the elephant in room with any JVM based big data architecture, which is the unmitigated shit that is zookeeper
    While I doubt that's what he was on about, what's your problem with ZK?
    Yeah that's absolutely not what I was on about. I'm not ops. At best I dip into a dev ops role when needed. Zookeeper's never been a problem for me because I don't manage it, I just know it's a thing that's there juggling servers or something.

    As a dev I'm a consumer of the services once they're up. So, for example, apache servers (ew) and solr, which is basically a shitty elasticsearch with shitty enterprisey configuration that nobody wants to use. We do use it because it works, and if massive xml files for configuration are your thing, you won't get my distaste. An ELK stack, on the other hand, is actually fun. It may be that Solr wins some performance contest or has all the MEPS, but I doubt it would matter to us. Similarly from my perspective Rabbit's advantages outweigh SNS as I see the latter introducing complexity I have to manage and an SDK that's worse than the AMQP library I have used.

    I figured the reason Solr is what it is, is a combination of semi neglected foss management and a b2b audience that that accepts what's handed to them. So I figured kafka is the same. This is speculation on my part, I fully expect someone to tell me I'm wrong for xyz reasons, or lazy for wanting things to be easy. Also the latest Solr is better, but still. Bleah.
    Kafka is a bit more unique than solr vs elasticsearch (elasticsearch).

    In type of server, it's more of a properties file, unix server style of java server layout, and can easily be integrated into linux services or docker containers, or whatever.

    In practice, you deploy a cluster, with probably at least three nodes for a production deployment (sometimes a few more). It will at least need two processes, kafka and zookeeper, even on a single node deployment. Its a binary protocol for producers and consumers, and some rules about how to partition messages across the cluster you want to enqueue. It's main, fairly novel feature, is the fact that all the messages to a topic across a cluster are indexed and searchable in parallel, and can either be persistent, or windowed by size or age, and this allows many interesting ways to process data at scale, as the consumers keep a bookmark, so can go back and rewind, and pick back up where they crashed, and all other kinds of semantics that are useful. It's mostly about scale.

    The last point is why you put up with the complexity. Kafka came out of linkedIn. It is fairly (or will be until soon, cause everyone wants a "not java" kafka) unique, but as I said, I'm prejudiced against zookeeper so I've been figuring out a new approach to the same problem. Different compromises for different purposes. Stream processing is really where all of this comes into it's own, so streamdb ksql, things like leveldb implementations built on it are really excellent, highly scalable solutions to what is inevitably a data ingestion problem, but can often then offer various 'real-time' (heh) insights into data.

    Quote Originally Posted by Frug View Post
    it ended up consuming ops cycles in a way that completely outstripped its position as a dependency to a tool we want to use.
    We had one main ops guy and one devops guy able to keep it up with no problems while also supporting mesos, consul, our servers, and whatever else. I was there for a couple of years with this setup.
    Things like transaction volume have a way of making some software completely hard to use in some cases where they would be very easy in others. A single mysql server directly connected to a page with no caching or anything is the obvious canonical example. We had also been using it, in some sense, since 2007 or so. Once our Kafka cluster had been in production for a bit, and the entire business grew, which increases clients, etc, etc, at some point we hit a point where zookeeper began to get flaky. I mean, the team responsible handle a lot of other stuff too. My point was that zookeeper began to consume a disproportionate amount of time, from a causing problems and "Blinking lights" point of view.

    E: It's also worth pointing out that most of my corporate hadoop experience was with a certain vendor who don't really use the Apache implementation, so different issues but no zookeeper.
    meh

  2. #2402

    Join Date
    April 14, 2011
    Posts
    5,589
    Nah the problem there is that Solr is ooollld. The first release of Lucene was 1999 and is something of a Doug Cutting Special in API design terms (see also: Avro, MapReduce). Likewise Solr is from the deep dark ages of the early 2000s, while Yonik is a fucking genius there's no getting away from the XML.

    That said it is a similar story. The only functional area where ES will beat Solr is rapidly evolving data, which almost no one uses it for. Otherwise Solr is just a better product. There are some ease of use areas where ES is nicer*, but that stops mattering when you're seriously looking at needing dozens of heterogenous machines in a tiered config to meet your throughput needs and your consistency model basically stops existing all because web developers don't want to work with schemas.

    *And frankly not that many, Solr 6/7 have closed almost all the gaps.

    Quote Originally Posted by erichkknaar View Post
    ...My point was that zookeeper began to consume a disproportionate amount of time, from a causing problems and "Blinking lights" point of view.
    It's worth noting that as of the newer releases Kafka has almost eliminated the ZK dependency. It's still there but the load is on the order of 1% of what it used to be. Unless you insist on using it for tracking consumer offsets like some kind of dinosaur.
    Last edited by elmicker; October 29 2017 at 07:55:51 PM.

  3. #2403
    Donor erichkknaar's Avatar
    Join Date
    April 10, 2011
    Posts
    9,126
    Quote Originally Posted by elmicker View Post
    Nah the problem there is that Solr is ooollld. The first release of Lucene was 1999 and is something of a Doug Cutting Special in API design terms (see also: Avro, MapReduce). Likewise Solr is from the deep dark ages of the early 2000s, while Yonik is a fucking genius there's no getting away from the XML.

    That said it is a similar story. The only functional area where ES will beat Solr is rapidly evolving data, which almost no one uses it for. Otherwise Solr is just a better product. There are some ease of use areas where ES is nicer*, but that stops mattering when you're seriously looking at needing dozens of heterogenous machines in a tiered config to meet your throughput needs and your consistency model basically stops existing all because web developers don't want to work with schemas.

    *And frankly not that many, Solr 6/7 have closed almost all the gaps.

    Quote Originally Posted by erichkknaar View Post
    ...My point was that zookeeper began to consume a disproportionate amount of time, from a causing problems and "Blinking lights" point of view.
    It's worth noting that as of the newer releases Kafka has almost eliminated the ZK dependency. It's still there but the load is on the order of 1% of what it used to be. Unless you insist on using it for tracking consumer offsets like some kind of dinosaur.
    I know. That said, I have't looked at it since April, so...
    meh

  4. #2404
    Frug's Avatar
    Join Date
    April 10, 2011
    Location
    Canada
    Posts
    13,228
    Quote Originally Posted by elmicker View Post
    Nah the problem there is that Solr is ooollld.
    It's obvious that it's old. I knew the moment I was forced to start using it, I was like "jesus how old is this".


    there's no getting away from the XML.
    Putting SQL and javascript transformers in an XML file is inexcusable!

    Otherwise Solr is just a better product.
    I highly suspect "better" becomes immediately subjective based on what you want. It's only just got nested documents, while you can dump what you want into ES with no problem. IMO the documentation is garbage, maybe you don't think so, but having to use it has caused us numerous pains. I have yet to see any reason that would make it "better".

    Edit: Correction, this oddity that the elk docker image requires max_map_count set on the host (well, increased from default) to even run is a bit silly. I've never seen anything else require modifications to the host to run. Considering that's memory use related you may have a point.
    Last edited by Frug; October 30 2017 at 12:09:35 AM.

    Quote Originally Posted by Loire
    I'm too stupid to say anything that deserves being in your magnificent signature.

  5. #2405
    Movember '11 Best Facial Hair, Best 'Tache Movember 2011Movember 2012Donor helgur's Avatar
    Join Date
    April 24, 2011
    Location
    Putting owls in your Moss
    Posts
    8,482
    https://slproweb.com/products/Win32OpenSSL.html ragepost in red is an amusing read

  6. #2406
    Daneel Trevize's Avatar
    Join Date
    April 10, 2011
    Location
    T L A
    Posts
    12,069
    Shining Light Productions is a sole proprietorship
    Really? You'd never know from the Product Support section on that linked page, explaining a 0-tolerance policy for typos "since you ARE e-mailing a real developer"...
    Quote Originally Posted by QuackBot View Post
    Idk about that, and i'm fucking stupid.

  7. #2407
    Movember '11 Best Facial Hair, Best 'Tache Movember 2011Movember 2012Donor helgur's Avatar
    Join Date
    April 24, 2011
    Location
    Putting owls in your Moss
    Posts
    8,482
    Not really an uncommon occurance to have these sort of primadonna tendencies among developers though.

    Should have called it «Bacon of Shining light» for added comedy

  8. #2408
    Daneel Trevize's Avatar
    Join Date
    April 10, 2011
    Location
    T L A
    Posts
    12,069
    Oh subversion. 'Repos can be smaller than working copies because repos compress diffs'.
    Sure, if it's 100MB vs 250MB of text. But not when a wc's 4.6GB for that same repo...
    "Vacuum pristine copies". Apparently added to cleanup as automatic in 1.8, then optional in 1.9.
    Boom, bloat nuked.
    Quote Originally Posted by QuackBot View Post
    Idk about that, and i'm fucking stupid.

  9. #2409

    Join Date
    April 14, 2011
    Posts
    5,589
    Quote Originally Posted by Daneel Trevize View Post
    Oh subversion.
    I've spotted the root cause of your problems.

  10. #2410
    Daneel Trevize's Avatar
    Join Date
    April 10, 2011
    Location
    T L A
    Posts
    12,069
    It's not like this old repo/wc was used often, what with not noticing such bloat and all.
    svn is still fine for central work, but git is damned potent elsewhere. Just haven't got around to learning the cli for it, SourceTree works well enough instead.
    Quote Originally Posted by QuackBot View Post
    Idk about that, and i'm fucking stupid.

  11. #2411
    Donor Sponk's Avatar
    Join Date
    April 10, 2011
    Location
    AU TZ
    Posts
    11,322
    SourceTree works well enough
    This is literally the first time I have heard that statement.
    Contract stuff to Seraphina Amaranth.

    "You give me the awful impression - I hate to have to say - of someone who hasn't read any of the arguments against your position. Ever."


  12. #2412
    Donor halbarad's Avatar
    Join Date
    April 9, 2011
    Posts
    4,818
    Quote Originally Posted by Sponk View Post
    SourceTree works well enough
    This is literally the first time I have heard that statement.
    I find it works well for viewing the branch graph, but that's just because I haven't bothered setting any aliases for the git log features (or spent any time learning all the options for it).

    Actual commit/push/pull/etc is either done in the command line or using VSCode (mostly depending on what I'm committing).

  13. #2413
    Donor Sponk's Avatar
    Join Date
    April 10, 2011
    Location
    AU TZ
    Posts
    11,322
    I mostly use it for resolving conflicts by picking and choosing blocks.
    Contract stuff to Seraphina Amaranth.

    "You give me the awful impression - I hate to have to say - of someone who hasn't read any of the arguments against your position. Ever."


  14. #2414

    Join Date
    May 31, 2011
    Posts
    3,330
    Yeah, SourceTree for the "Lemme have a closer look at this" stuff. Common commit/push/merge etc. tasks in the CLI.

  15. #2415
    Donor Aea's Avatar
    Join Date
    April 13, 2011
    Location
    Colorado
    Posts
    14,062
    Quote Originally Posted by Sponk View Post
    SourceTree works well enough
    This is literally the first time I have heard that statement.
    I've been using it for a year now since GitX stopped receiving updates for the same purpose. Is fine.

  16. #2416
    Frug's Avatar
    Join Date
    April 10, 2011
    Location
    Canada
    Posts
    13,228
    Quote Originally Posted by Aea View Post
    Quote Originally Posted by Sponk View Post
    SourceTree works well enough
    This is literally the first time I have heard that statement.
    I've been using it for a year now since GitX stopped receiving updates for the same purpose. Is fine.
    Looks interesting, must give it a look.

    Good gravy, we've been forced to use visual studio team services and I can't imagine a worse tool. The UI is consuming time from people like nothing else could. I've seen entire days destroyed as if they never happened. Mein gott.

    Quote Originally Posted by Loire
    I'm too stupid to say anything that deserves being in your magnificent signature.

  17. #2417
    Donor halbarad's Avatar
    Join Date
    April 9, 2011
    Posts
    4,818
    Quote Originally Posted by Frug View Post
    Quote Originally Posted by Aea View Post
    Quote Originally Posted by Sponk View Post
    SourceTree works well enough
    This is literally the first time I have heard that statement.
    I've been using it for a year now since GitX stopped receiving updates for the same purpose. Is fine.
    Looks interesting, must give it a look.

    Good gravy, we've been forced to use visual studio team services and I can't imagine a worse tool. The UI is consuming time from people like nothing else could. I've seen entire days destroyed as if they never happened. Mein gott.
    Which part of the UI is the problem? The work items, builds and releases are pretty good. The code bit isn't great but I usually only use that to grab the clone url and put in PRs.

  18. #2418

    Join Date
    April 14, 2011
    Posts
    5,589
    I was forced to use VSTS earlier this year. It was surprisingly not shit. The build system in particular actually looked pretty good.

  19. #2419
    Frug's Avatar
    Join Date
    April 10, 2011
    Location
    Canada
    Posts
    13,228
    Quote Originally Posted by halbarad View Post
    Which part of the UI is the problem? The work items, builds and releases are pretty good. The code bit isn't great but I usually only use that to grab the clone url and put in PRs.
    So far everything. We use it primarily for task management and it's just awful. Nothing is discoverable, people constantly have to ask for help on how to do things that ought to be simple. It's particularly bad on a small screen and everyone's on mac books.

    There isn't even a way to format text as code in a story or a task. I thought this was for developers, how is a code formatting button not a thing? How does it not support markdown? Why don't any time estimates roll up to any kind of summary on the parent object? Why do bugs need sprints? Why do we need a plugin to support a query on the kanban board? How is someone expected to use a board without filtering stories somehow? It's madness, madness I say. Also it's ugly. So ugly. The way items pop up in modal windows half the time, and then they stack on top of eachother with no visual indication that they're doing that. It's 2017 and everything still follows the 'click the save button to save' paradigm in this thing.

    It's gotten better after managers put time and effort into things like adding plugins, building pages and queries for people and giving us all a talk on how to use this monstrosity, but we all wish we didn't have to. Of the 20 people i know of who have to use it, maybe 3 like it. I miss Target Process, and I also hated that at the time.

    Build/release stuff is probably fine but our code actually doesn't live in vsts atm. Don't ask why. At least I don't have to use it for code reviews. I highly suspect it won't have all the cheeky emoji options of... everything else on the planet.

    Quote Originally Posted by Loire
    I'm too stupid to say anything that deserves being in your magnificent signature.

  20. #2420
    Donor halbarad's Avatar
    Join Date
    April 9, 2011
    Posts
    4,818
    Quote Originally Posted by Frug View Post
    Quote Originally Posted by halbarad View Post
    Which part of the UI is the problem? The work items, builds and releases are pretty good. The code bit isn't great but I usually only use that to grab the clone url and put in PRs.
    So far everything. We use it primarily for task management and it's just awful. Nothing is discoverable, people constantly have to ask for help on how to do things that ought to be simple. It's particularly bad on a small screen and everyone's on mac books.
    That's a fair point about small screens, some things really don't scale well for it and minimising the side bars don't help too much as it'll just reappear when you move pages.

    There isn't even a way to format text as code in a story or a task. I thought this was for developers, how is a code formatting button not a thing? How does it not support markdown? Why don't any time estimates roll up to any kind of summary on the parent object? Why do bugs need sprints? Why do we need a plugin to support a query on the kanban board? How is someone expected to use a board without filtering stories somehow? It's madness, madness I say.
    There is filtering and searching on kanban boards and backlogs, you should also be able group stories by who they are assigned to.

    Lack of Markdown support is a problem for lots of people, so hopefully we'll see that in the next few months, especially with the wiki that's in markdown and all the other bits of markdown support they're adding elsewhere in the product.

    Estimates not rolling up has been raised by a few people I've talked with about it. I haven't seen any mention of that changing but I would hope the product team are aware of it.

    Also it's ugly. So ugly. The way items pop up in modal windows half the time, and then they stack on top of eachother with no visual indication that they're doing that. It's 2017 and everything still follows the 'click the save button to save' paradigm in this thing.
    It definitely isn't the prettiest UI. Click to save is pain but I doubt it's likely to change soon.

    It's gotten better after managers put time and effort into things like adding plugins, building pages and queries for people and giving us all a talk on how to use this monstrosity, but we all wish we didn't have to. Of the 20 people i know of who have to use it, maybe 3 like it. I miss Target Process, and I also hated that at the time.
    It does suffer somewhat from being a very generic product but thankfully the extension engine is pretty good and they keep adding more, recently with a few more options around work items. Writing extensions for it is also pretty easy since it's usual typescript with some json configuration docs.

    Build/release stuff is probably fine but our code actually doesn't live in vsts atm. Don't ask why. At least I don't have to use it for code reviews. I highly suspect it won't have all the cheeky emoji options of... everything else on the planet.
    It'll pull code from pretty much any git repo the build agents can access (usually private for more complex builds).

    Code review is not bad in the PR process. Not sure it supports emoji there yet but I can find out.

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •